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Executive Summary 

In January 2019, the Internal Revenue Service (IRS) Free File program (“Free File” or 
“the program”) opened for its 17 th filing season. Established in 2002, the IRS Free File program 
makes free online tax preparation and electronic filing available to 70 percent of the taxpaying 
population; for Filing Season (FS) 2019, this equated to taxpayers with an adjusted gross income 
(AGI) of $66,000 or less. The program is made possible through a partnership between the IRS 
and the tax software industry. The partnership is governed through a mutually negotiated 
Memorandum of Understanding (MOU) between the IRS and the Free File Alliance (FFA) 1 —a 
coalition of individual private-sector tax preparation companies 2 —that has evolved over the 
program’s lifespan to address issues as they have arisen. 

While cumulatively the program has resulted in more than 53 million free returns e-filed and an 
estimated $1.6 billion in savings to taxpayers 3 , the Free File program has also faced criticism in 
recent years as questions have been raised by Congress and the press about low annual 
participation rates and assertions of inadequate oversight, deceptive business practices, and 
misaligned roles for government and industry. This year’s launch came amid criticisms of the 
program and speculation that industry partners were not acting in the best interest of taxpayers. 

In response to these assertions as well as recent oversight activities and congressional inquiries, 
IRS leadership determined that an independent assessment was necessary to ensure the continued 
operations and integrity of the IRS Free File program. Because the MITRE Corporation operates 
the Service’s federally funded research and development center (FFRDC), the IRS engaged 
MITRE to conduct an independent and objective assessment of the IRS Free File program in its 
current state, providing recommendations for improvements to the program, and by extension to 
the MOU that governs the program. 

Methods 

Beginning in June 2019, MITRE established several workstreams for addressing each of the 
targeted assessment areas reflected in Figure ES-1. Taken together, these workstreams produced 
findings and recommendations related to Free File member compliance with the MOU and 
potential enhancements to the program. Highlights of the assessment approach include: 


1 In 2014, the Free File Alliance officially changed the name of the organization to Free File, Inc. (FFI) doing business as the 
Free File Alliance. While FFI is the listed name on all agreements and MOUs beginning in 2014, the two names are used 
interchangeably. The IRS references the Alliance on IRS.gov “Free File: About the Free File Alliance,” and the organization has 
not changed the name of its own website: https://ffeefilealliance.org/. This document will use the term Free File Alliance, the 
Alliance, or FFA except when FFI is used in a specific context or quote. 

2 Industry participation in the Free File program is voluntary. As stipulated in the MOU, there is a process for applying for 
membership to the Free File Alliance and companies may leave the program at their discretion. Also outlined in the MOU, IRS 
and FFA leadership can remove members from the program if there are certain MOU violations. Through FS 2019, the Free File 
Alliance included 12 member companies; currently there are only 11 participating members of the program. As of July 2019, 
Drake [1040.com] chose to withdraw from the program and will not be a part of it going forward. Since they were a part of the 
program for FS 2019, they are still included in this assessment. The decision to leave came from the member itself and to 
MITRE’s knowledge were not related to any MOU infractions. 

3 Internal Revenue Service, "IRS, Free File Alliance announce changes to improve program; Improved taxpayer options available 
for 2019 Free File program," Internal Revenue Service, 2 November 2018. [Online], Available: 

https://www.irs.gov/newsroom/irs-ffee-file-alliance-announce-changes-to-improve-program-improved-taxpayer-options- 
available-for-2019-free-file-program. [Accessed 2 July 2019]. 
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• Current Environmental Scan. MITRE conducted an environmental scan to better 
understand the Free File program in terms of historical context, taxpayer participation, the 
role of IRS and industry in its public-private partnership (PPP), the economics of “free” as 
it pertains to Free File, and recent interest in the program. 

• Free File Program Analysis. MITRE conducted stakeholder interviews—including IRS 
leadership—and performed document reviews to assess the IRS Free File program’s 
alignment to objectives, performance, and the IRS management and oversight functions. 

• FFA Member Organization Interviews (“site visits”). MITRE conducted interviews 
with representatives from each Free File member organization to collect information on 
their interactions and agreements with the IRS regarding the Free File program and 
applicable business processes, practices, and techniques for how each Free File member 
detennines if a taxpayer qualifies for Free File. Each of the interviews followed a similar 
approach: standard interview protocols, targeted questions (both related to the IRS Free 
File program as well as the MOU Article 4, Standards of Practice) and in-person or phone 
interviews with participants from each Free File member organizations (selected by the 
member company’s leadership). 

• Web Forensics Analyses. MITRE performed an analysis to detennine if any coding 
within the member’s website influences search engine routing. In addition, MITRE 
analyzed web search query traffic metrics for internet searches that signaled an intent to 
find free tax filing services, based on a representative sample of organic (unpaid) search 
keywords for free tax filing. This included a click-through analysis ofIRS.gov site usage 
metrics to investigate where taxpayers went from the IRS Free File web portal. This 
analysis was also used to detennine how often taxpayers searched for free tax filing 
services (especially during tax filing season) using two primary search engines, Google 
and Bing; to what site(s) these users went following their searches; and the differences in 
search engine results. 

• Taxpayer Journey. MITRE conducted an expert review of the IRS Free File portal as 
well as each individual Free File member offer. This review was twofold in its objective: 
first, MITRE evaluated the 12 member sites for compliance with MOU requirements 
(including an initial landing page analysis); second, MITRE evaluated the 12 member sites 
and the Free File-related pages on IRS.gov against recognized software user interface 
usability principles (called heuristics) in order to identify potential usability problem 
areas. For both parts, MITRE set-up user accounts and ran scenarios to test the sites in a 
systematic, exhaustive fashion. MITRE’s usability assessment used both industry- 
recognized criteria (heuristics) as well as MITRE’s interpretation of how MOU 
requirements translate into usability criteria and whether sites are optimized for mobile 
devices. 

• Taxpayer Usability Testing. MITRE conducted a study to test the usability of the IRS 
Free File portal and assess taxpayers’ actual experience in filing tax returns using three 
randomly selected Free File software offerings. MITRE conducted in-person usability 
testing sessions with 29 taxpayers who filed their taxes electronically using Free File 
software in prior year filings. During the testing, taxpayers were observed as they 
completed fictitious tax scenarios using the IRS Free File portal and Free File software. In 

PRE-DECISIONAL - FOR INTERNAL IRS W&l USE ONLY - NOT FOR PUBLIC RELEASE 


IV 



PRE-DECISIONAL - FOR INTERNAL IRS W&l USE ONLY - NOT FOR PUBLIC RELEASE 


addition to data on the usability of the Free File services, MITRE collected and analyzed 
data on the frequency of errors committed, taxpayer satisfaction, and experience (via a 
brief questionnaire), and taxpayers’ perceptions of the Free File program. 

• Legal Analysis. MITRE’s General Counsel, in coordination with outside counsel, 
conducted a review of the MOU’s terms and conditions to determine recommended 
revisions to the MOU to more clearly define the parties' responsibilities to enhance and 
promote the IRS Free File program, address any shortcomings in the MOU language as it 
relates to the prohibition against unfair methods of competition, and provide enforcement 
capability via liability clauses if the MOU's terms and conditions are breached by a 
Free File member organization. 

• Recommendations. Based on the data gathered from the workstreams, MITRE developed 
recommendations for overall improvements to the Free File program, potential 
enhancements to the MOU, and improvements to usability ofIRS.gov and some aspects of 
member offerings. 

Figure ES-1 shows how each of these workstreams collectively contributed to addressing 
targeted assessment areas (assessment objectives) including: documenting the current Free File 
program environment, assessing Free File program management and oversight, assessing FFA 
member compliance with the MOU, assessing taxpayer experience with respect to the program, 
and ultimately drafting recommendations for the program and MOU. Outputs from some 
workstreams contributed to more than one assessment area. 



Document Current 
Free File Environment 


Z Trends 

Z Free File Market 
Z External Literature 



Assess Free File Program 
Management & Oversight 




Assess Free File Alliance (FFA) 
Member Compliance 




Taxpayer Experience 


Z Program Objectives 
V Program Management 
Z IRS Compliance Approach 
Z FFA Compliance Approach 


V IRS Data Analysis 
Z Web Analysis 
Z Taxpayer Journey 
Z FFA Leadership and 
Member Interviews 


Z Taxpayer Journey 
Z Taxpayer Usability Testing 



Develop Program & MOU 
Recommendations 


Z Legal Review 
Z SME Review 
Z IRS Interviews 
Z FFA Leadership and 
Member Interviews 
Z Workstream Findings 


Figure ES-1. High-Level Approach to Free File Program Assessment (Aligned to Assessment Objectives). 
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Addressing Public Perceptions 

Throughout the assessment, MITRE considered the perspectives of various media, oversight, and 
advisory organizations: the recent ProPublica articles on the Free File program, the 2018 Internal 
Revenue Service Advisory Council (IRSAC) Annual Report, and the 2018 National Taxpayer 
Advocate Annual Report to Congress 4 . Three primary themes emerged from these external 
sources: 

1. Member companies use deceptive strategies to steer taxpayers away from using the Free 
File program services towards fee-based services. 

2. The IRS Free File Program Office does not provide adequate oversight. 

3. Low usage rates signal that the Free File program is not working. 

However, MITRE’s findings from this assessment determined that while there may be an 
element of truth to each of the criticisms, the data indicate the issues are much more nuanced and 
don’t tell the full story. 

MITRE conducted the assessment of the Free File program in its current state—the tax software 
industry providing free tax preparation and filing services under a MOU with the IRS. It was not 
in the scope of MITRE’s effort to address the underlying political issue that is the genesis of 
much of the controversy surrounding the Free File program—whether the IRS should create its 
own free tax preparation and filing program or continue to rely on the tax software industry to 
provide these services. While MITRE recognizes that government services provided to its 
citizens are continually evolving, and that Congress may determine in the future that the IRS 
should offer that service (and provide it with the appropriate funding to do so), for the purposes 
of this study, MITRE assumes that industry will continue to be the entity that provides free tax 
return preparation and filing offerings to taxpayers. All of MITRE’s recommendations in this 
report are based on that assumption. 

Search Engine Avoidance 

The issue that seems to have created the most publicity and was at the center of much of the 
congressional interest is the practice of coding that caused members’ Free File landing pages to 
be excluded from an organic search through Google or Bing search engines. During FS 2019, 
five of the 12 FFA members did engage in a practice that excludes their company’s Free File 
landing page in organic searches. When questioned about it, most members contended this keeps 
them in compliance with the MOU (language in the MOU preamble) and in alignment with IRS 
marketing that states Free File is “accessible only through IRS.gov.” 5 MITRE did not take a 
position on the practice—determination of whether this is a deceptive business practice is not 
within MITRE’s legal purview. The perception points to the need for the IRS to take a position 
on the practice and incorporate new language into the MOU. MITRE also conducted an in-depth 
analysis of what the results of organic searches provided. 


4 Internal Revenue Service, "Internal Revenue Service Advisory Council Public Report, Publication 5316, Catalog Number 
71824A," Internal Revenue Service, Washington, D.C., 2018.; Taxpayer Advocate Service, "National Taxpayer Advocate Annual 
Report to Congress," Taxpayer Advocate Service, Washington, D.C., 2018.; Series of ProPublica articles from March 20, 2017 to 
June 5, 2019, available at https://www.propublica.org 

5 Internal Revenue Service, "Changes strengthen program; Free File can also help navigate new tax law provisions," Internal 
Revenue Service, 11 january 2019. [Online]. Available: https://www.irs.gov/newsroom/irs-free-file-opens-today-in-advance-of- 
tax-season. [Accessed 10 September 2019], 

PRE-DECISIONAL - FOR INTERNAL IRS W&l USE ONLY - NOT FOR PUBLIC RELEASE 


vi 



PRE-DECISIONAL - FOR INTERNAL IRS W&l USE ONLY - NOT FOR PUBLIC RELEASE 



• Five of the 12 members used a coding device 6 to keep 
their Free File landing page out of organic searches. 


Member companies use deceptive 
strategies to steer taxpayers away from 
using the Free File program services 
towards paid commercial services 
(ProPublico). 


• Based on a representative sample of organic (unpaid) 


search keywords for free tax filing, the highest 
number of searches went to IRS.gov (>780,000). The 
top member landing page had 698,000 searches. The 
next highest was less than 180,000 searches. The 


total unpaid traffic to the 12 members was 1.8 
million. 

• Paid searches resulted in the highest amount of 
traffic (using the same representative sample of 
keywords as above), more than 10 million total to the 
12 member sites combined, none to IRS.gov (IRS does 
not pay for advertising). 

• IRS.gov Free File landing page received 2.8 million 
entrances 7 (the landing page was the first page a user 
saw when arriving on IRS.gov) during FS 2019. 

• Four of the five members who engaged in the 
practice had increases in Free Filing in FS 2019, and 
one remained relatively flat. Five of the seven who 
did not engage in the practice had increases in Free 
Filing, and two had decreases. 

• Number of taxpayers who received free filing outside 
of Free File through free commercial offerings is not 
publicly available. However, according to the FFA, the 
current members provided a total of more than 17 
million of these free commercial filings in FS 2019 8 . 


6 “<META NAME="ROBOTS" CONTENT="NOINDEX">meta 

7 Because the IRS does not employ any tracking codes once a user leaves the site, there is no way to know how many of these 2.8 
million entrances became Free File users. 

8 The actual number of free offerings may be higher. One company who was a member in 2019 did not provide a number. 
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Table ES-1 depicts the effects of organic versus paid searches for the top 108 non-branded high 
value keyword search terms that drove traffic to Free File member landing pages. This data was 
derived from website traffic information downloaded from S EM rush 1 ', a digital market software 
as a service company. 


Table ES-1. Comparing Paid Search Traffic to Organic Search Traffic 9 . 


FFI Member Sites 

(Member names redacted) 

Est Traffic to 

FFI Landing Pages 

Est Traffic to 

Non-FFI Landing Pages 

Est Paid Traffic to 
Non-FFI Landing Pages 

Est Cost 

for Paid Traffic 


0 

3,823 

246,856 

$416,912 


378 

248 

0 

0 


648 

65 

0 

0 


0 

11 

54,594 

$91,616 


0 

0 

0 

0 


3,370 

0 

0 

0 


0 

697,756 

5,400 

$103,270 


3,488 

95,688 

1,812,769 

$3,060,979 


205 

179,161 

3,662,118 

$6,213,910 


11,696 

562 

0 

0 


36,310 

9,547 

3,039,163 

$5,125,896 


0 

40,759 

1,446,217 

$2,426,933 

Total 

56,095 

1,812,218 

10,267,117 

$17,347,900 

IRS.gov (for comparison) 

700,780 

83,819 

0 

0 


The data in Table ES-1 shows that for the organic searches, more traffic went to the IRS.gov 
Free File landing page than to any one of the individual member sites’ non-Free File landing 
pages. However, the most traffic, as would be expected, went to paid advertising. Seven of the 
12 Free File members bought ads for keywords related to free tax filing that directed traffic to 
the landing pages for their non-Free File free tax filing software offering, resulting in five times 
more traffic than organic searches. The data used for this analysis (not shown in this table) 
indicated that no member had paid advertising for their Free File landing pages. This is not 
unexpected, as the MOU specifically assigns advertising responsibility to the IRS. However, 
during the FFA member interviews, one member stated that their company does employ 
advertising practices for the Free File program 9 10 . 

Program Oversight 

The perception that members were undertaking potentially deceptive practices that resulted in 
vulnerable taxpayers paying for tax services-compounded by the program’s low participation 
rates-led to criticism that the IRS Free File Program Office was not providing adequate oversight 
of member behavior. What this critique overlooks is the fact that the Free File program operates 
as a public-private partnership and, as such, program oversight is a mutual, collaborative effort 


9 SEMrush Inc., "All-in-one Marketing Toolkit," SEMrush Inc., 2019. [Online], Available: https://www.semmsh.com/. [Accessed 
week of 27 May 2019], 

10 During MITRE interviews with representatives of FFA member organizations, one member indicated that their company pays 
for advertising of the Free File program through a number of methods. However, the interview did not include detailed discussion 
of advertising practices specific to key word search. For this analysis, MITRE looked specifically at ads purchased for key words 
related to free tax filing. The results of this analysis indicated that no purchased ads directed users to the landing pages of Free 
File offers. 
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with the industry, represented by the Free File Alliance. While there is room for improvement, 
based on assessment findings, MITRE considers the IRS’s current compliance processes 
adequate and effective to support the integrity of the program. In part, because of this long¬ 
standing partnership with FFA and continuous assessment and improvement of the program as it 
has evolved. 



• Because the Free File program is a PPP, and not a 
paid contractual relationship, it must work 
collaboratively with industry on oversight of the 
program. 


The IRS Free File Program Office does 
not provide adequate oversight (IRSAC, 
Taxpayer Advocate Service [TAS], 
members of Congress). 


• IRS program office conducts member website 
reviews, evaluates member offer coverage and 
monitors member acceptance rates, while working 
closely with FFA to address, document, and resolve 
any identified or potential compliance breaches. 


• IRS's existing agency governance of the Free File 
program effectively allows the program to fulfill the 
IRS's currently articulated program goals. 


• While the program office does not have updated, 
measurable objectives for internal agency 
performance, its shared oversight of member 
compliance strictly follows the provisions of the MOU 
and meets joint program objectives. 


Taxpayer Participation in Free File Program 

Finally, what has led many to question the overall effectiveness of the Free File program is the 


participation numbers themselves. In 2018, fewer than 3 million filers used the Free File program 
to file their tax year 2017 returns out of nearly 104 million taxpayers that meet the eligibility 
requirements for the program. However, these numbers don’t tell the entire story and take the 
issues of taxpayer choice and taxpayer behavior out of the equation. The Free File program will 
generally appeal to those taxpayers that prefer a “do-it-yourself’ (DIY) method of tax 
preparation and filing. Looking retrospectively at the eligible population from FS 2018 and how 
those taxpayers chose to file can help the IRS better understand their target population for Free 


File. 
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What's been said... What the data show... 


Low usage rates signal that the Free 
File program is not working (TAS, 
ProPublica, members of Congress). 


• While 103.8 million taxpayers were technically 
eligible to use Free File in FS2018, half of those used a 
paid preparer, taking them out of the do-it-yourself 
(DIY) tax preparation pool in that year. 

• Another 9.8 million taxpayers in the DIY pool received 
refund anticipation checks (RACs) or refund 
anticipation loans (RALs), taking them out of the 
eligible pool. 

• Though paper filers are a potential target population 
for Free File, the 8.8 million DIY taxpayers who filed 
on paper removed themselves from the 2018 
candidate pool. 

• Three million taxpayers went to Volunteer Income 
Tax Assistance (VITA) sites, removing them from the 
candidate pool. 

• The actual number of filers using DIY software and 
eligible to use Free File in FS 2018 was about 30 
million (see Figure ES-2). Of those, about 9 percent 
used Free File. 

• Of those 30 million, a significant percentage (17.7 
million for FS 2019 according to FFA) received free 
commercial filing from the current members. 

• The VITA program (the bricks and mortar assisted 
free equivalent of Free File) had approximately 3 
million taxpayers use their service VITA has 
thousands of volunteer preparers, and more than 
2,000 partner organizations who conduct outreach 
for their 11,000 sites nationwide. 


As shown in Figure ES-2, once the number of taxpayers that choose to file via a different 
method—e.g., with a paid preparer or through a Volunteer Income Tax Assistance (VITA)—or 
elects to receive their tax refund immediately through a refund anticipation loan (RAL) or a 
refund anticipation check (RAC)—are removed due to their preferences, the actual pool of 
candidates for the Free File program from 2018 is about 30 million taxpayers. A significant 
percentage of those receive free filing outside the Free File program. Though the numbers for 
free commercial filings are unknown for tax year (TY) 2017, a study by the FFA in 2015 showed 
it to be 19 million for that year from the 14 members, and the FFA attested to 17.7 million for 
TY 2018 from 11 current members. 
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103,778,000 Eligibile Taxpayers - TY 2017 



■ Paid Preparers 

■ RAC/RAL self-prepared 

■ Paper filers - self-prepared 

■ VITA/TCE 

■ Free Filers 

■ Potential Add'l Free Filers 


Figure ES-2. Break-down of 103,778,000 Taxpayer Eligible for Free File (TY2017). 

It is outside the scope of this assessment for MITRE to determine what the proper percentage of 
Free File taxpayers should or could be. Without further analysis and research into taxpayer 
behavior it is impractical to make that type of determination. Presumably if awareness was at 
peak saturation, participation would be greater than it is now. However, there are many 
competing free offers in the market that even taxpayers who are aware of Free File may gravitate 
to. The IRS’s ability affect the Free File members’ behavior in terms of their commercial 
offerings, or taxpayers’ behavior in terms of how they choose to file their return, is limited. What 
the IRS does have control over is what and how much they do to ensure program awareness. 

Free File Member Assessment Results - MOU Compliance 

Recent attention and criticisms notwithstanding, the Free File program has operated for 17 years 
under the governance of an MOU—an agreement that all members must abide by or be removed 
from the program. The IRS asked MITRE to conduct a fact-finding review of each member to 
determine whether members were in compliance with the most recent version of the MOU at the 
time of the assessment. 

To assess MOU compliance, MITRE drew findings from IRS interviews and document reviews, 
interviews with Alliance leadership and an Alliance-contracted reviewer, and fact-finding 
interviews with representatives from each of the Free File member organizations. In addition, 
MITRE incorporated the results of the hands-on taxpayer journey reviews of member Free File 
landing pages and websites, as well as results from the web forensics analyses where applicable. 

Overall, based on the various assessment workstreams, all Free File members were compliant 
with the MOU, in particular with Article 4, Standards of Practice. 
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Article 4 outlines level of service, software and website functionality, security, and disclosure of 
forms and schedules. In addition, it includes taxpayer service options and other requirements 
such as innovation. MITRE found two instances of MOU non-compliance and an inconclusive 
finding for nine requirements for some or all members. There were also two requirements that 
were not applicable at the time of review for several members. 

The two non-compliance instances were: 

• One member was non-compliant with a sentence in MOU 4.15.4. The MOU states that 
“Members must clearly list their free customer options ... on the Member’s Free File 
Landing Page (or such page must have a clear and prominent link to such disclosures 
directly from this page).” Based on MITRE’s landing page analysis, the link was not 
“clear and prominent” on their landing page 11 . 

• One member was non-compliant with MOU 4.32.4. The MOU requires members to 
“communicate not less than once annually via email with their taxpayer customers who 
used Free File services and completed their returns through Free File in the immediately 
preceding tax year.” The representative from the member acknowledged they were 
supposed to initiate an email to their prior tax season Free File taxpayers and self-attested 
that they did not provide the required email prior to the tax season. 

While MITRE deems these to be non-compliant issues, none rise to the level of severity or 
concern regarding the integrity of the Free File program. 

Taxpayer Experience Analysis 

In addition to assessing compliance with MOU requirements, MITRE also conducted research 
activities to understand the taxpayer experience using the Free File program. These research 
activities were designed to represent the perspective of the end user of the Free File system—the 
taxpayer. As such, MITRE identified pain points and areas for improvement to make the 
experience easier, clearer, and more accurate for taxpayers. This included an expert review of 
IRS.gov and the 12 member sites for potential usability issues, and usability testing interviews 
with actual taxpayers. 

Taxpayer Journey Analysis 

MITRE conducted a taxpayer journey analysis to evaluate potential usability issues with the Free 
File member systems and Free File-related web pages. The taxpayer journey analysis also 
evaluated member landing pages for compliance with the MOU, as described above. The rest of 
the analysis was focused on the taxpayer experience from a usability perspective and should not 
be interpreted as evidence for or against member compliance with the MOU. 

The taxpayer journey analysis used both established industry criteria and MOU requirements 
related to system usability and user expectations for system functionality as anchor points for the 
analysis. Specifically, MITRE used the following sets of criteria to conduct the review of the 12 
member sites and IRS.gov’s Free File-related web pages: 


11 As of the date of this report, the member has updated their Free File landing page. 
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• Nielsen’s 10 usability heuristics 12 . These 10 criteria, called “heuristics,” are widely 
recognized measures of system usability. Taken together, the 10 heuristics construct a 
measure of user experience and include factors such as “match between system and the 
real world,” “error prevention,” “flexibility and ease of use,” and “consistency and 
standards.” 

• MOU requirements, from a usability perspective (user experience [UX] MOU 
Adherence). MITRE examined MOU requirements pertaining to the user experience and 
constructed usability criteria from these MOU requirements. These criteria assess user 
interface elements closely associated with MOU requirements and included factors such as 
“understands offerings available” and “transparency around ineligibility.” MITRE’s 
assessment against these criteria are not included in the evaluation of MOU compliance 
presented in Section 4.0. 

• Mobile experience. MITRE evaluated the extent to which the examined sites were 
optimized for mobile device use. 

For all three of these groups of criteria, the objective of the evaluation was to identify potential 
usability issues and prioritize them via a 1-5 (1 = worst, 5 = best) severity score. 

MITRE’s analysis found that, while the Free File members adhere to many UX best practices as 
measured by Nielsen’s usability heuristics, they could improve the most in “error prevention” 
and “match between system and real world” heuristics. With respect to examining the sites in 
tenns of a usability interpretation of MOU requirements, the area where members have the most 
need for improvement was “transparency around ineligibility.” 

All of the scores from the review are rolled up into a dashboard that shows a quick view of the 
summary scores for user experience, UX MOU adherence, and mobile experience, with a more 
detailed breakout of the scores from the heuristic review by each heuristic and grouping of MOU 
requirements (see Section 5.1). Figure ES-3 shows the summary dashboard from the taxpayer 
journey heuristic analysis, and aggregates Nielsen’s 10 heuristics under “user experience” issues 
related to the usability in the MOU under “UX MOU adherence,” and whether or not the site is 
mobile-optimized under “mobile experience.” The 12 member sites are shown in the columns by 
number and the heuristic dimensions are displayed down the rows. 


12 Nielsen’s 10 Usability Heuristics: Nielsen, Jakob. (1994). Heuristic evaluation. In Nielsen, J., and Mack, R.L. (Eds.), Usability 
Inspection Methods, John Wiley & Sons, New York, NY 
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Dimension 


User Experience 


UX MOU Adherence 


Mobile Experience 


DDDBDDB 





♦ ♦ t # ♦ t 4 
******* 
******* 


8 9 10 11 12 


* * * * * 

* * * * * 

* * * * * 


SUMMARY 

At a glance, the Free File providers adhere 
to many UX best practices. Overall, they 
could improve the most in “Error 
Prevention” and “Match Between System & 
Real World” heuristics. The most significant 
UX MOU improvement to make across the 
board is “Transparency around ineligibility.” 


Score range 


85% to 100%. Area of strength. 

^ 70% to 84%. Room for improvement. 

0% to 69%. Area of weakness. 


Figure ES-3. Heuristic Evaluation Dashboard Summary. 

The top two highest-rated members both had exceptional user experience, intuitive mobile 
experiences, and followed the MOU’s guidance related to taxpayer experience and usability. 
Other members might consider mirroring the transparency Member 10 has around Free File 
eligibility, which lets the user know quickly whether or not they are eligible to proceed through 
the Free File process. 

Two members received the lowest ratings for the usability of their user interfaces. Three 
members did not have mobile-optimized sites, which would make it very difficult for users to file 
their taxes from a mobile device. 

Taxpayer Usability Analysis 

MITRE conducted the taxpayer usability study via in-person interviews with 29 taxpayers who 
previously used Free File from selected zip codes in Chicago. To select an interview location, 
MITRE reviewed the density of TY 2018 Free Filers in multiple clusters of zip codes in U.S. 
metropolitan areas; Chicago had a relatively high volume of Free Filers, which allowed MITRE 
to send invitation letters to potential participants and elicit a sufficient interview sample. MITRE 
randomly selected a subset of Chicago Free Filers to receive invitation letters to participate. 

The objectives of these interviews were to understand how taxpayers locate the Free File 
program options; understand how taxpayers select a software provider; capture usability issues 
with IRS websites related to the Free File program; capture usability issues with specific vendor 
platforms; and understand taxpayers’ previous experiences with the Free File program. During 
the sessions, participants completed fictitious scenarios to evaluate the experiences of completing 
three tasks: 1) locating the Free File program online; 2) selecting a Free File software offer, 
including with the software lookup tool; and 3) completing a tax return. MITRE randomly 
selected three Free File member software offers to test with taxpayers. Each participant filed a 
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fictitious return with one of the three selected offers. Taxpayers used mocked-up information 
(e.g., fictitious Form W-2s and other documentation) to complete the tasks. 

MITRE identified and reported all usability issues observed by the research team with each of 
the three Free File usability tasks and provided suggested solutions for addressing each of these 
issues individually. Drawing on the data collected from the usability tasks, analysis of qualitative 
data gathered during debriefing and with semi-structured interview questions, the satisfaction 
and user experience questionnaires, and metrics on taxpayer behaviors and selections during the 
tasks, MITRE identified the several high-level findings. When it comes to navigating to the Free 
File program and selecting a member offer, taxpayers: 

• Lacked understanding of what the Free File program is and its relationship to the IRS 

• Struggled to find the Free File landing page, including through IRS.gov 

• Experienced difficulties and confusion with selecting a software offering, particularly 
when evaluating Earned Income Tax Credit (EITC) eligibility 

• Expressed that they were overwhelmed with choices and information 

• Used strategies beyond eligibility criteria to select software offers 

With respect to using member-specific Free File program tax preparation software, common 
usability issues included trouble finding where to report adjustments to income, difficulty 
choosing between filing statuses, and trouble navigating different systems’ “flow.” 

Overall, taxpayer ratings of ease and satisfaction varied by task, vendor, and taxpayer 
demographic characteristics. 

Findings and Recommendations 

Building upon the analysis from the various workstreams in this assessment, MITRE provides 
the following findings and recommendations for improvements to the Free File program and to 
the Free File MOU that governs the program. The findings and recommendations are grouped by 
key themes that emerged through the execution of the MITRE assessment. The recommendations 
include actions that the IRS may take to improve the integrity and effectiveness of the Free File 
program. 

Where appropriate, MITRE provides workstream-specific recommendations the body of this 
assessment. 
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1. Objectives of the Free File program have not been updated since the 
program’s inception. 

Findings 

Recommendations 

The IRS and FFA have not revisited the 
objectives of the program since its 
inception. The objectives (reflected in the 
MOU) are outdated and don’t reflect 
leadership’s current view of program 
success. 

The program objective of providing the 
venue for free tax filing for 70 percent of 
the population has been met. The e-filing 
objective has been met. No other metrics 
currently exist by which the program office 
can measure its performance. 

MITRE recommends that IRS leadership, in 
collaboration with the FFA, define new program 
objectives and create metrics to determine the 
Program Office’s performance against those 
objectives. Recognizing that the IRS and FFA 
will need to negotiate any formal changes to 
joint program objectives, MITRE recommends 
that IRS develop internal objectives and metrics 
that reflect its own priorities. While agreement 
between the IRS and FFA would be the ideal, 
translating priorities into objectives and metrics 
should not be contingent on that agreement. 

For example, according to IRS leadership, its 
focus for free file is less about target number of 
participants, and more about maximizing 
awareness to ensure people recognize it is a 
choice. 

Potential metrics could include: 

• Number of companies participating, 
potentially targeting a specific market 
share 

• Actual number of Free File taxpayer 
candidate pool/participants (e.g., xx% of 
the DIY community minus RALs/RACs) 

• Metrics associated with compliance (e.g., 
offering parameters, etc.) 

• Awareness numbers (see additional 
recommendations) 

• Web analysis metrics (e.g., number of 
clicks on the Free File link on the 

IRS.gov home page, percentage of select 
keyword searches that bring taxpayers to 
the IRS.gov landing page, etc.) 
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2. If Free File objectives are revised to focus on taxpayer awareness, more 
needs to be known about the target population and taxpayer behavior. 

Findings 

Recommendations 

The IRS has collected demographics data 
on users of Free File and migration data of 
those users, but they do not have 
demographics data of the potential pool of 
Free File users. This data would be helpful 
in targeting outreach, (see Section 3.3) 

MITRE recommends the IRS conduct data 
analysis of the demographics of the population 
who are prime candidates for Free File but not 
using it. 

The total population pool of 103 million 
taxpayers who are eligible may not 
represent an accurate picture of the pool of 
taxpayers who are candidates for Free File. 

Taxpayers make choices that take them out 
of the candidate pool. About 9 percent of 
taxpayers who have not taken themselves 
out of the candidate pool by (for example) 
using a paid preparer or choosing a RAC or 
RAL, use Free File (not taking into 
consideration those who choose free 
commercial offerings) There is no 
consensus on what would be an acceptable 
number. 

MITRE recommends the IRS conduct a taxpayer 
behavior study to better understand the factors 
involved in a taxpayer’s choice of filing 
methods. Understanding taxpayer motivation 
and choices could help determine what would be 
an acceptable Free File number, if a number is 
needed at all. This aligns with the IRS-related 
priorities of the internal MITRE research 
program and will be proposed for a future 
research project. 

While each individual member receives 
customer feedback in various ways for 
their own Free File product, the IRS has 
not conducted a customer survey since 

2009. 

MITRE recommends the IRS conduct a 
customer survey of Free File users specifically to 
the IRS experience of Free File (not the software 
they filed their return through) to use as a 
baseline and create an online survey for users to 
capture continuous feedback. 

The IRS has no access to data that would 
indicate whether a DIY filer paid for a 
return or received a free commercial 
offering. As the issue is whether a low- 
income taxpayer has to pay for filing, 
knowing how many taxpayers receive free 
filings either through Free File or a 
commercial product would give the IRS a 
better understanding of their target 
markets. 

MITRE recommends, that in the spirit of 
transparency, the IRS propose the industry to use 
an indicator on free commercial filings 

Or 

MITRE recommends the IRS request that FFA 
provide them the aggregate number of taxpayers 
who file their taxes using free commercial 
products each year. When developing metrics, 
this could help with understanding a more 
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accurate picture of the population of the 

Free File candidate pool. 

Leadership acknowledged that the number 
of returns submitted through Free File have 
not increased in the same way that e-file 
numbers have and articulated a goal of 
increased awareness. The last awareness 
study was conducted in 2011. 

MITRE recommends the IRS commission an 
awareness study to baseline taxpayer awareness 
of Free File, specifically of targeted 
demographics detennined in the prior 
recommendation. 

MITRE further recommends, based on the 
results of this study—and other recommended 
research—the IRS conduct a targeted awareness 
campaign to the desired Free File demographics, 
and include other IRS-sponsored venues such as 
low-income tax clinics (LITCs) and VITA sites, 
including IRS’ own website. 

Paid advertising yields results. In MITRE’s 
web analysis of those searching for free tax 
filing, an estimated 85 percent of the traffic 
followed a paid search result to member 
sites, while the other 15 percent of visits to 
member sites came from an organic search 
result. Searchers were more than five times 
likely to click on a paid search result. 

MITRE recommends the IRS conduct a cost- 
benefit analysis to determine whether to 
purchase paid advertising to reach taxpayers 
searching for free tax filing. We recognize the 

IRS may have to seek statutory authority and 
budget-specific funding for such advertising if it 
was deemed beneficial. 


3. IRS Free File program is a long-standing public-private partnership, but 
there is a lack of public awareness about what that means to the program. 

Findings 

Recommendations 

Free File program is a partnership between 
the IRS and industry that requires 
cooperation to achieve success. This 
partnership requires a balance that serves 
the interests of the government and 
taxpayers, but also creates a value to the 
for-profit industry that provides the 
service. 

The IRS Free File program could benefit 
from providing clarity and scope around 
the PPP, both internally and externally. 

The purpose and scope of PPPs are not 
common, and awareness is key to success. 

MITRE recommends that a health assessment of 
the PPP become part of the overall metrics for 
the Free File program, using a generally 
accepted framework provided by external 
sources, such as that used by MITRE in this 
assessment. 

MITRE further recommends that the IRS 

Free File Program Office incorporate talking 
points addressing the unique nature of the PPP in 
tenns of joint governance and the value a PPP 
provides to all partners as central themes when 
addressing critics or publicly discussing the 
program to help address this lack of awareness. 
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4. Rules that were put into the MOU to promote fairness among FFA 
members and promote choices for taxpayers translate into unclear 
presentation of offers and confusion for taxpayers. 

Findings 

Recommendations 

The FFA is obligated to provide Free File 
to eligible taxpayers through individual 
commercial sites such that, when taken in 
aggregate these services are offered to the 
lowest 70 percent of the taxpayer 
population, calculated using AGI. 

Moreover, the MOU requires that each 
company must offer services to at least 10 
percent of total eligible taxpayers but to no 
more than 50 percent of taxpayers (“10/50 
rule”)—a requirement set to level the 
playing field for smaller providers. 

While the 10/50 rule has been (and may 
continue to be) beneficial to FFA members 
to ensure equity, it’s possible that it has 
outlived its usefulness. MITRE did not 
have adequate time, nor was it within 
scope, to do an analysis of how the 10/50 
rule impacts Free Filed returns. However, 
from a taxpayer perspective, the different 
company offerings create confusion as a 
taxpayer navigates the various choices. 

MITRE recommends the IRS conduct an 
analysis/study of the 10/50 rule. This could 
include a confidential market study of the 
members to help understand the economic 
incentive to assist in preserving competition 
amongst the members and promoting taxpayer 
economic welfare; a small pilot to detennine the 
business impact of removing the rule; a 
consideration of whether differing rules for the 
two large companies would benefit or create 
inequity; and/or an in-depth data analysis of how 
the different offerings have impacted members’ 
free file numbers. 

Even if the rule is not changed, MITRE 
recommends simplifying the display of 
members’ offering to reduce taxpayer confusion. 


5. Usability activities—both with taxpayers and by usability experts— 
provide insight into how taxpayers experience various aspects of the 

Free File program. 

Findings 

Recommendations 

MITRE conducted a complete heuristic 
assessment of the Free File website and 
members websites, and a usability 
assessment with Free File users. Both 
assessments found that improvements 

A complete list of recommendations to improve 
usability and the user experience on IRS.gov is 
provided in the Taxpayer Experience Results 
Report (Appendix G) and taxpayer journey 
supplemental materials (Appendix F). While 
usability-specific, these recommendations 
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could be made to enhance the user complement the program-level recommendations 

experience. above and as such may be repetitive. 

MITRE recommends that IRS: 

• Standardize how Free File is referenced on 
IRS platfonns and communications 

• Take steps to increase ease of understanding 
eligibility requirements 

• Ensure that taxpayers understand that 
determining eligibility for a software offer is 
ultimately their responsibility 

• Recommend best practices to increase 
usability of members’ tax preparation 
software systems 

• Conduct research on the usability taxpayer 
access and use of the Free File system 

• Conduct research on Free File awareness, 
attending to different demographic groups’ 
awareness 

• Eliminate or change the EITC eligibility 
question in the lookup tool and on the 
software offers page 

o If the IRS stated on the Free File site that 
all taxpayers eligible for EITC are 
eligible for Free File, the question could 
be removed from the tool. MITRE 
recognizes this would be a topic for 
negotiation between the IRS and FFA 
because of how that could impact some 
companies’ appearance in the tool. 

o If the IRS chooses to leave EITC in the 
lookup, MITRE recommends linking the 
question to the EITC eligibility tool, but 
ONLY after conducting usability testing 
on the EITC eligibility tool and 
addressing the results. 
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6. The MOU is an evolving document that is added to with each new 
negotiation, resulting in a piecemeal agreement that is out of date in some 
areas. 

Findings 

Recommendations 

The MOU generally has provisions added 
with each revision, but few are taken out. 
Much of the same language is used, not 
only in the objectives, but also in the 
technology portion, that has become 
outdated (e.g., text CAPTCHA). 

MITRE recommends that the IRS and the 

Alliance jointly detennine new, measurable 
objectives for the Free File program, and draft a 
new MOU, pulling and using provisions from 
the current MOU that are relevant, discarding 
provisions that are not, and creating new 
provisions based on the current environment. 

All but two of the members’ Free File 
offerings are the same as their commercial 
offerings. The security/privacy rules of 

Free File are the same as e-File, and all 

Free File members are also members of the 
Security Summit where the IRS works 
collaboratively with the industry to 
improve security. 

MITRE recommends simplifying the MOU in its 
security and privacy provisions by simply 
requiring members to adhere to the e- 
File/Security Summit requirements and 
recommendations. 

MITRE’s review of the individual 
members’ processes for applying the 

Free File indicator (without looking at the 
actual coding) indicated most members 
used the same process that should place the 
indicator only on Free Filed returns. 
However, that is not always the case. 

In MITRE’s data analysis of returns 
marked with the Free File indicator, 
several thousand returns from one vendor 
indicated the presence of a RAC. The 
vendor noted that the IRS infonned them 
of this irregularity, and they discovered a 
coding error had mistakenly placed the 
indicator on returns that had not been Free 
Filed, which was subsequently corrected. 
Another vendor also had several hundred 
returns with the Free File indicator on 
returns with AGI of more than $66,000. 

MITRE recommends adding language to the 

MOU to specify conditions under which the 
indicator is applied to ensure quality control. 

Though not necessarily dependent on the MOU, 
MITRE also recommends the IRS develop a 
quality control method that would allow the IRS 
to validate returns with the Free File indicator 
were actually Free Filed. 
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7. Search exclusion techniques are the latest new issue for consideration 
when entering into the next round of MOU negotiations. 

Findings 

Recommendations 

Five of the companies used a NOINDEX 
code to exclude their company’s Free File 
landing page from organic searches. They 
reported that their belief was that this kept 
them compliant with the MOU. However, 
while the MOU states the IRS will provide 
links to their site, it does not prohibit 
taxpayers from accessing their sites 
directly. IRS press releases state that 
taxpayers can ONLY access Free File sites 
through IRS.gov. 

MITRE recommends that if the IRS wants 
taxpayers to go to Free File sites only through 
IRS.gov, the MOU should reflect that 
specifically as a provision, as opposed to the 
current reference in the preamble. 

MITRE further recommends the IRS detennine 
their position on the use of NOINDEX and add a 
provision to the MOU addressing that use. 


PRE-DECISIONAL - FOR INTERNAL IRS W&l USE ONLY - NOT FOR PUBLIC RELEASE 




PRE-DECISIONAL - FOR INTERNAL IRS W&l USE ONLY - NOT FOR PUBLIC RELEASE 


Table of Contents 

Executive Summary .iii 

1.0 Introduction.1 

1.1 Background.2 

1.1.1 How Taxpayers Access and Use Free File.3 

1.1.2 How the Free File Program Works: IRS and Industry Partners.5 

1.1.3 Economic Drivers of Industry.9 

1.1.4 Recent Interest in the Free File Program.9 

1.2 Scope.11 

1.3 Assumptions and Limitations.12 

1.4 Organization of This Document.14 

2.0 Methods Overview.15 

2.1 Current Environmental Scan.17 

2.2 Free File Program Analysis.18 

2.3 Free File Alliance Member Organization Interviews (Site Visits).19 

2.4 Web Forensics Analyses.20 

2.5 Taxpayer Journey.22 

2.6 Taxpayer Usability Testing.26 

2.7 Legal Analysis.28 

3.0 Eligible Taxpayers and Actual Participation.29 

3.1 Overall Trends.29 

3.2 Taxpayer Behavior and Awareness of the Free File Program.30 

3.3 A Closer Look at Eligibility and Participation.31 

PRE-DECISIONAL - FOR INTERNAL IRS W&l USE ONLY - NOT FOR PUBLIC RELEASE 

xxiii 

























PRE-DECISIONAL - FOR INTERNAL IRS W&l USE ONLY - NOT FOR PUBLIC RELEASE 

3.4 Free File and VITA.33 

4.0 Compliance with the MOU.35 

4.1 MOU Provisions Set Criteria for Compliance.35 

4.2 Free File Alliance Member Adherence to MOU.39 

4.2.1 FFA Member Assessment Results: Overview.39 

4.2.2 Emerging Issues: Search Techniques.42 

4.2.2.1 How Taxpayers Find the Free File Program.42 

4.2.2.2 Search and MOU Compliance.45 

4.3 Assuring Adherence to MOU.47 

4.3.1 Program Objectives.47 

4.3.2 Program Structure.50 

4.3.3 Member Oversight and Compliance Activities.55 

4.4 Assessment of Public-Private Partnership.58 

4.5 MOU Renewal in 2021.61 

5.0 Taxpayer Experience with Free File.63 

5.1 Taxpayer Journey.64 

5.1.1 Heuristic Review of Member Free File Websites.64 

5.1.2 Heuristic Review of IRS.gov.67 

5.2 Taxpayer Experience Usability Findings.71 

5.2.1 General Free File Program Findings and Usability Issues.72 

5.2.2 Locating the Free File Program Online.72 

5.2.3 Selecting a Free File Software Offer, Including with the Software Lookup Tool... 73 

5.2.4 Completing a Tax Return Using a Specific Free File Offer.75 

PRE-DECISIONAL - FOR INTERNAL IRS W&l USE ONLY - NOT FOR PUBLIC RELEASE 


XXIV 

























PRE-DECISIONAL - FOR INTERNAL IRS W&l USE ONLY - NOT FOR PUBLIC RELEASE 


5.3 Integrated Look at Taxpayer Experience.76 

6.0 Recommendations.79 

6.1 Improvements to the Free File Program.79 

6.1.1 Define Updated Program Objectives and Create Corresponding Metrics.80 

6.1.2 Take Steps to Better Understand the Free File Target Taxpayer Population and 

Behavior.80 

6.1.3 Increase Awareness of the Free File as a Long-Standing PPP.82 

6.1.4 Simplify Member Offerings to Reduce Taxpayer Confusion.82 

6.2 Recommendations Related to the MOU.83 

6.2.1 Update MOU to Reflect Current Environment.83 

6.2.2 Determine the IRS Position on Search Exclusion and Provide Guidance.84 

6.2.3 Legal Analysis of MOU.85 

6.3 Taxpayer Experience-Specific Recommendations.86 

Appendix A The Economics of IRS Free File.A-1 

Appendix B Crosswalk of External Program Reviews.B-l 

Appendix C ProPublica Articles on IRS Free File Program.C-l 

Appendix D Initial Assessment: Overall Search Engine Results.D-l 

Appendix E How the Major Web Search Engines Work.E-l 

Appendix F Taxpayer Journey.F-l 

F. 1 Heuristic Review of Free File Web Pages.F-l 

F.2 Heuristic Review ofIRS.gov.F-l 

Appendix G Taxpayer Experience Testing Report.G-l 

Appendix H Acronyms and Initials.H-l 

Appendix I References.1-1 

PRE-DECISIONAL - FOR INTERNAL IRS W&l USE ONLY - NOT FOR PUBLIC RELEASE 


XXV 


























PRE-DECISIONAL - FOR INTERNAL IRS W&l USE ONLY - NOT FOR PUBLIC RELEASE 


Table of Tables 

Table 1. Taxpayer Eligibility Criteria by Free File Alliance Member (as of August 2019).8 

Table 2. Document Organization.14 

Table 3. Stakeholder Interview Participants.19 

Table 4. Free File Program Agreements and MOUs.37 

Table 5. Top Sources of Traffic to the Free File Member Landing Page.42 

Table 6. Estimated Traffic Distributions from Organic and Paid Search Result to Landing 
Pages/Estimated Cost for Paid Traffic.44 

Table 7. Free File Program Office Key Areas of Focus.52 

Table 8. Public-Private Partnership Assessment Criteria for Free File Program.59 

Table of Figures 

Figure 1. Free Options for DIY Tax Filing: Free File Program and Free Commercial Offerings.. 4 

Figure 2. High-Level Approach to Free File Program Assessment (Aligned to Assessment 
Objectives).17 

Figure 3. Nielsen's 10 Usability Heuristics.24 

Figure 4. Heuristics to Evaluate MOU Adherence from a Usability Perspective.25 

Figure 5. Free File Accepted Returns, FY 2003-2018.30 

Figure 6. Economic Cost by Filing Method.32 

Figure 7. Break-down of 103,778,000 Taxpayer Eligible for Free File (TY 2017).33 

Figure 8. Comparison of Two Free Programs: VITA (Preparer-assisted) and Free File (DIY)... 34 

Figure 9. Top Keywords for Member Free File Landing Pages.43 

Figure 10. Seasonality of Free File-Related Searches.45 

Figure 11. The “MOU-specified” Path of Free File Through IRS.gov.46 

PRE-DECISIONAL - FOR INTERNAL IRS W&l USE ONLY - NOT FOR PUBLIC RELEASE 


XXVI 




















PRE-DECISIONAL - FOR INTERNAL IRS W&l USE ONLY - NOT FOR PUBLIC RELEASE 


Figure 12. Key IRS Free File Program Management and Oversight Activities for TFF.57 

Figure 13. Public-Private Partnership: Balance of Value to Stakeholders.58 

Figure 14. The MOU as a Negotiation Tool.62 

Figure 15. Severity Level and Description.65 

Figure 16. Heuristic Evaluation Dashboard Summary.65 

Figure 17. Heuristic Evaluation Dashboard.66 

Figure 18. Samples from Heuristics Review for IRS.gov Landing Page.67 

Figure 19. Design Alternative #1 to Improve Scanning for FFA Member Offers.68 

Figure 20. Design Alternative #2 to Improve Scanning for FFA Member Offers.69 

Figure 21. Observed Taxpayer Paths to Reach Free File.78 


PRE-DECISIONAL - FOR INTERNAL IRS W&l USE ONLY - NOT FOR PUBLIC RELEASE 

xxvii 













PRE-DECISIONAL - FOR INTERNAL IRS W&l USE ONLY - NOT FOR PUBLIC RELEASE 


1.0 Introduction 


The IRS Free File program is a 17-year old program that makes free online tax preparation 
and electronic fding available to 70 percent of the taxpaying population, specifically those 
whose adjusted gross income is $66,000 or less. The program is made possible through a 
partnership between the IRS and the tax software industry. The partnership is governed 
through a mutually negotiated Memorandum of Understanding that has evolved over the 
program’s lifespan to address issues as they have arisen. 

While cumulatively the program has resulted in approximately 53 million free returns e-filed 
and an estimated $1.6 billion dollars in savings to taxpayers 1 , the Free File program has also 
faced criticism in recent years as questions have been raised by Congress and the press about 
low annual participation rates and assertions of inadequate oversight, deceptive business 
practices, and misaligned roles for government and industry. Considering this attention, IRS 
leadership asked MITRE to conduct an independent and objective assessment into the current 
state of the program, and to make recommendations. To conduct this assessment, MITRE 
assembled a team to take a comprehensive look into multiple aspects of the program; 
including, but not limited to, industry compliance with the current MOU, IRS program 
management and oversight activities, emerging issues around web search and avoidance 
techniques, what is known about the eligible population and participation rates, and taxpayer 
experience with industry websites and relevant IRS.gov pages. 

Throughout this document there are some reoccurring key themes based on our findings that 
may be helpful to keep in mind, including: 

• Compliance with the current MOU is only one part of the story. While member 
compliance with the MOU is critical, too narrow a focus on that compliance may 
overlook nuances of the program that are worth considering. Taking a broader view of 
free options for tax filing may provide a more holistic view of Free File program—with 
regards to the IRS, industry partners, and the taxpayers they serve—and potential 
enhancements for the future. 

• Free File program is a partnership between IRS and industry that depends upon 
balance and cooperation to achieve success. Any improvements the IRS wants to 
make to the program need to be mutually negotiated and enforced by both sides of the 
partnership. This partnership requires a balance that serves the interests of the 
government and taxpayers, but also creates a value to the for-profit industry that 
provides the service. 

• More needs to be known about taxpayer choice and filing behavior when it comes 
to participation—or lack of participation—in the Free File program. 


In January 2019, the Internal Revenue Service (IRS) Free File program (“Free File” or “the 
program”) opened for its 17 th filing season (FS), providing access to free online tax preparation 
and filing to eligible taxpayers through a long-standing partnership with the tax software 
industry. This year’s launch came amid criticisms of the program and speculation that industry 
partners were not acting in the best interest of taxpayers. In response to these assertions as well 
as recent oversight activities, IRS leadership determined that an independent assessment is 
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necessary to ensure the continued operations and integrity of the IRS Free File program. As the 
Service’s federally funded research and development center (FFRDC), the IRS engaged MITRE 
to provide IRS, Wage and Investment (W&I) with an independent and objective assessment of 
the IRS Free File program in its current state, providing recommendations for improvements to 
the program, and by extension to the Memorandum of Understanding (MOU) that governs the 
program. 

The intended outcome of this effort is that the IRS will ensure continued equitable application 
and integrity of the Free File program through improved situational awareness of adherence to 
the Free File MOU, potential amendments to the MOU based on emerging issues and legal 
analysis, potential program improvements, and overall taxpayer experience with the program. 

As the operator of the Center for Enterprise Modernization FFRDC, sponsored by the 
Department of the Treasury and the IRS, MITRE is an independent, objective, conflict-free 
organization that has no commercial interest in the Free File Alliance (FFA), its members, or the 
Free File program. This message was communicated to all interview and usability testing 
participants to encourage candid, truthful responses and to engender trust between the team and 
the participants. 

1.1 Background 

The Free File program was established in 2002—implemented during FS 2003 though 
partnership between the IRS and the tax software industry— allowing access to online software 
that provides free online tax preparation and filing services to eligible moderate- and low-income 
taxpayers. The program came about as a direct result of the Office of Management and Budget’s 
(OMB) 2001 Quicksilver Task Force that established 24 e-government initiatives designed to 
improve govemment-to-govemment, government-to-business, and government-to-citizen 
electronic capabilities 13 . In accordance with this OMB directive, and as part of a separate effort 
to increase individual taxpayer filings of electronic returns (e-file), the IRS worked with industry 
to envision the program, and subsequently entered into a multi-year partnership agreement with a 
collection of private sector tax software companies who came together to form the Free File 
Alliance, LLC 14 (FFA or “the Alliance”). As part of that agreement, in exchange for the private 
industry offering free services, “the federal government has pledged to not enter the tax 
preparation software and e-filing services marketplace.” 15 

In 2014, the Free File Alliance officially changed the name of the organization to Free File, Inc. 
(FFI) doing business as (dba) the Free File Alliance. While FFI is the listed name on all 
agreements and MOUs beginning in 2014, the two names are used interchangeably. This 


13 Internal Revenue Service, "Free File: About the Free File Alliance," Internal Revenue Service, 25 March 2019. [Online]. 
Available: https://www.irs.gov/e-file-providers/about-the-free-file-alliance. [Accessed 2 June 2019]. 

14 The IRS references the Alliance on IRS.gov “Free File: About the Free File Alliance”, and the organization has not changed 
the name of its own website: https://freefilealliance.org/ 

15 In each MOU, through the 8 th MOU of October 31, 2018. For more information, see Internal Revenue Service, "Free File: 
About the Free File Alliance," Internal Revenue Service, 25 March 2019. [Online]. Available: https://www.irs.gov/e-file- 
providers/about-the-free-file-alliance. [Accessed 2 June 2019]. 
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document will use the term Free File Alliance, the Alliance, or FFA except when FFI is used in a 
specific context or quote. 

While the number of taxpayers filing electronically has greatly increased since the program’s 
inception—almost 88 percent of all individual tax returns were e-filed in fiscal year (FY) 2017 
— the use of the Free File program has remained stagnant for the past several years. About 2.5 
million people filed returns using Free File software in FY 2017 compared to over 3 million in 
FY 2014, and the peak of about 5 million taxpayers in tax year (TY) 2004 16 . Recent data from the 
IRS Compliance Data Warehouse (CDW) database shows that Free File numbers started to pick 
up since TY 2016. Compared to TY 2017, approximately 74,000 more returns have been filed to 
date for TY 2018. These trends are discussed in more detail in Section 3.1. 

1.1.1 How Taxpayers Access and Use Free File 

The IRS Free File program is currently available to any taxpayer who has an Adjusted Gross 
Income (AGI) of $66,000 or less—or 70 percent of all individual taxpayers. The software is 
provided by participating Alliance members. For FS 2019 taxpayers chose from 12 companies’ 
Free File offerings. 17 For each available option, software is the do-it-yourself (DIY) variety, that 
is, questionnaire format (rather than form-based) that navigates the taxpayer through the tax 
filing process without the intervention of a tax practitioner, like a paid preparer or accountant 18 . 
State tax returns may be free or paid depending on the specific Free File offer. 

Ideally, users access Free File through IRS.gov 19 —either going to IRS.gov and clicking through 
to find the program’s landing page (the first page of a website or home page) or by navigating 
directly to the program’s IRS.gov landing page through a web search engine. The path taken by 
taxpayers looking for a DIY free option to file is shown in Figure 1. 


16 Taxpayer Advocate Service, "National Taxpayer Advocate Annual Report to Congress: Volume 1," Taxpayer Advocate 
Service, Washington, D.C., 2018.; Data Sources: IRS Data Book (FY 2017) and IRS, 2017 Filing Season Statistics (29 Dec 

2017) . In TY 2016, just under 2.3 percent of all eligible taxpayers submitted returns using the Free File software, and just 0.2 
percent of taxpayers used Free File Fillable Forms to submit their returns. IRS response to TAS infonnation request (7 Sept. 

2018) .7 

17 Currently there are only 11 members participating in the Free File program. As of July 2019, Drake [1040.com] chose to 
withdraw from the program. However, since they were a part of the program for FS 2019, they are still included in this 
assessment. 

18 In 2009, the IRS added FFFF for free filing. Unlike the traditional Free File program assessed here, the FFFF is not a 
questionnaire-based form, but resembles as fillable pdf that can be filed electronically free of cost. FFFF is available to all 
taxpayers regardless of income. It is an “unbranded” product provided to IRS by industry, but all customer service is the 
responsibility of IRS Free File program staff. While there is some mention of FFFF in this document, it is out of scope of the 
assessment. 

19 Internal Revenue Service, "Free File: Do Your Federal Taxes for Free," Internal Revenue Service, 24 June 2019. [Online]. 
Available: https://www.irs.gov/filing/free-file-do-your-federal-taxes-for-free. [Accessed 2 July 2019], 
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Taxpayer selects Taxpayer selects a 

Free File program Free File offer based Taxpayer files taxes for 

landing page on eligibility criteria free using IRS Free File 



Figure 1. Free Options for DIY Tax Filing: Free File Program and Free Commercial Offerings 20 . 

Once on the IRS.gov landing page, taxpayers are presented with a list of all Free File offerings 
and a brief description of each. Since each member is pennitted to have slightly different criteria 
for their offering (see Section 1.1.2), the IRS also provides a software look-up tool (also referred 
to as a wizard or “Guide Me To A Company” tool) to help taxpayers make a selection that works 
best for their situations. 

If taxpayers are eligible for Free File (for FS 2019 if AGI is $66,000 or less) and meet the 
specific requirements of the member offer they selected, taxpayers will be able to file their taxes 
for free through the Free File program. The selected offer will be subject to all the oversight and 
regulation outlined in the program MOU. 

There is another path that could also result in a free offering—though, importantly, not through 
the Free File program (and as, such, not subject to the program regulation). If taxpayers make 
more than the $66,000 ceiling, they are not eligible for the Free File program; however, many 
commercial software providers do offer free editions of their products apart from their Free File 
program offerings. These products may or may not offer the same features and choosing one of 
these products may or may not result in actually being able to file for free. These “free 
commercial products” are sometimes confused with Free File products, both anecdotally and by 
critics of the program. Only taxpayers that access a Free File offering (online software product) 
with a direct link from the IRS.gov landing page, are actually participants in the Free File 
program 

If taxpayers start at IRS.gov but discover that they have an AGI greater than $66,000, they may 
leave Free File to seek other alternatives. Similarly, as discussed in Section 4.1, if taxpayers 
begin using a Free File product and discover that they are not eligible for a specific offer 


20 Icons created by Vectors Point, Lil Squid and monkik from Noun Project. 
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(because of age or other criteria) there are safeguards in place to keep eligible taxpayers in the 
program. That is, taxpayers whose AGI is $66,000 or less but do not qualify for a specific 
program for other reasons will be sent back to the IRS.gov landing page to select another offer. 
Taxpayers who make more than $66,000 can legitimately be offered the commercial product by 
the FFA members, free or otherwise. 

1.1.2 How the Free File Program Works: IRS and Industry Partners 

The roots of the Free File program lie within two foundational political directives—both of 
which are also tied to a broader goal of increasing electronic filing. Taken together, the IRS 
response to these directives resulted in the partnership between the IRS and the tax software 
industry that exists today to provide free e-file services to taxpayers. These are: 

• Restructuring and Reform Act of 1998 (RRA 1998). 21 The IRS was asked to cooperate 
with the private sector to reach a target to have 80 percent of federal tax and information 
returns filed electronically by 2007, with the expectation that increased e-filing would 
improve tax compliance, lower operating costs, and reduce human errors in tax data 
transcription. 

• OMB Quicksilver Task Force (2001). 22 Part of President George W. Bush’s Management 
Agenda, the Free File initiative from this task force directed the IRS to provide free and 
secure online tax return preparation and filing services to taxpayers, in anticipation that 
reduced cost of tax preparation would increase e-filing. 

In 2002, Treasury Secretary Paul O’Neill asked then IRS 
Commissioner Charles Rossotti to partner with the private 
sector for its established expertise in electronic tax preparation 
and filing. 23 This is a critical piece of the program’s history in 
understanding the partnership that ensued. In directing the IRS 
to enter into this partnership, Treasury recognized the private 
sector could accomplish the directive more quickly and at a 
lower cost than the IRS. The IRS then relinquished any efforts 
to provide e-filing services and that became a primary 
provision of the agreement between the partners. The 
participating industry partners, in turn, assumed the role of 
providing free tax filing for the agreed-upon population of 
taxpayers. 

This arrangement led to the growth of a public-private partnership (PPP) between the IRS and 
the tax preparation software industry. While there is no consensus of how to define a PPP, 24 it is 


"[It is not the intent] for the IRS 
to get into the software 
business, but rather to open a 
constructive dialogue with those 
who already have established 
expertise in this field. In the end, 
this effort should come up with 
a better way to save time and 
money for both taxpayers and 
the Government" 

- Treasury Secretary Paul O'Neill 
January 30, 2002 


21 The Internal Revenue Service Restructuring and Reform Act of 1998, Pub.L. 105-206, 112 Stat. 685, enacted July 22, 1998. 

22 Internal Revenue Service, "Free File: About the Free File Alliance," Internal Revenue Service, 25 March 2019. [Online], 
Available: https://www.irs.gov/e-file-providers/about-the-free-file-alliance. [Accessed 2 June 2019], 

23 U.S. Department of the Treasury, Office of Public Affairs, "Treasury, IRS Announce New Efforts to Expand E-Filing," 
Washington D.C., 2002. 

24 M. Marsilio, G. Cappellaro and C. Cuccurullo, "The Intellectual Structure Of Research Into PPPs," Public Management 
Review, vol. 13, no. 6, pp. 763-782, 2011 
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generally defined as a cooperative agreement between two or more public and private sectors, 
typically of long-term nature. 25 MITRE discusses the IRS Free File Program Office in more 
detail in Section 4.3 and explores assessing a PPP in Section 4.4. 

Memorandum of Understanding: Documenting Roles, Rules, and Program Objectives 

The partnership described above is governed by a MOU between the IRS and the Free File 
Alliance, composed of individual tax software companies who apply for membership with FFA 
and who agree to follow the rules and regulations set forth in the MOU. Individual provisions of 
the MOU describe in detail the terms and conditions required for continued participation in the 
program—and ultimately for achieving the program’s objectives. An important distinction, the 
MOU is not a contract (i.e., between an organization and a set of vendors) but a mutually 
negotiated tool for governance agreed upon by the two partners. 

Participation in the partnership—by both the IRS and the individual companies—is voluntary. 
Software companies have come and gone throughout the years, and at least one company who 
participated in the program in FS 2019 will not be back in FS 2020. 

While the companies come together to fonn a single entity in the Alliance; outside the Alliance 
they are still individual companies—with varying sizes, resources, proprietary business models 
and priorities. These differences were observed during MITRE interactions with member 
companies during site visits and interviews. 

The influence of the historical factors described above can be seen in the Free File program 
objectives, as stated in the current MOU between the IRS and the Free File Alliance: 

• Make tax return preparation and filing easier and reduce the burden on individual 
taxpayers, particularly the economically disadvantaged and underserved populations 

• Support the IRS’s statutory goals of increased electronic filing, pursuant to the IRS 
Restructuring and Refonn Act of 1998 

• Provide greater service and access to the Services to taxpayers 

• Implement one of the proposals in the President’s Fiscal Year 2003 budget, specifically to 
encourage further growth in electronic filing by providing taxpayers the option to file their 
tax return online without charge using cooperation with, and encouraging competition 
within, the private sector 

While the wording may be slightly different from the original agreement establishing the 
program in October 2002, the objectives in the current MOU have not changed in nearly 17 
years, even though e-filing now accounts for 88 percent of all individual income tax filings. (See 
Section 4. lfor further discussion of how the original agreement between the IRS and FFA has 
changed over time and Section 4.3.1 for further discussion of Free File program objectives.) 

Also included in the MOU—just below the objectives—is the agreement that IRS will not pursue 
its own tax preparation and filing software platform: “In recognition of this commitment, the 
federal government has pledged not to enter the tax preparation software and e-filing services 


25 G. A. Hodge and C. Greve, "On Public-Private Partnership Performance: A Contemporary Review," Public Works 
Management & Policy, vol. 22, no. 1, pp. 55-78, 2016 
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marketplace.” 26 This statement has also not changed in the 17-year program history, and program 
supporters and critics alike have mentioned this as a foundational element of the current Free 
File program. 

Free File Offerings: Why Taxpayer Eligibility Differs Member to Member 

The primary criterion for taxpayer eligibility in the program is AGI 27 . That said, the program is 
structured in such a way that individual members can set additional eligibility rules, including 
limitations on age, state, Earned Income Tax Credit (EITC) eligibility, etc. As discussed in 
Section 5.0, Taxpayer Experience with Free File, the difference in eligibility can sometimes be 
confusing to taxpayers. The reason for the variety comes from the program structure agreed upon 
in the MOU and provisions written into the agreement to encourage competition among 
members and provide choice to taxpayers 28 . 

In 2002, the initial agreement between the IRS and the FFA stipulated that Free File be made 
available to at least 60 percent of U.S. taxpayers—with no restriction on income. 29 30 After three 
filing seasons, the IRS and the FFA extended the agreement and amended it to limit that program 
to taxpayers with an AGI equal to or less than 70 percent of all taxpayers 31 , in order to include 
those “least able to afford e-filing tax returns, based upon verifiable characteristics in their tax 
return.” The IRS and the Alliance have maintained the 70 percent, AGI-based target ever since 32 . 

Importantly, the 70 percent taxpayer coverage requirement is a combined obligation across all 
members of the Alliance, and not for each individual member alone. In other words, the FFA is 
obligated to provide Free File to eligible taxpayers through individual member Free File sites 
such that, when taken in aggregate these services are offered to the lowest 70 percent of the 
taxpayer population, calculated using AGI. 

In addition, the MOU requires that each company must offer services to at least 10 percent of 
total eligible taxpayers but to no more than 50 percent of taxpayers—a requirement set to level 
the playing field for smaller providers (referred to in this document as the “10/50 rule”). Finally, 
also in aggregate, the MOU stipulates that every eligible taxpayer must be eligible for at least 


26 Internal Revenue Service and Free File, Incorporated, "Eigth Memorandum of Understanding on Service Standards and 
Disputes Between the Internal Revenue Service and Free File, Incorporated," Internal Revenue Service, 2018. 

27 The IRS defines Adjusted Gross Income as “as gross income minus adjustments to income.” Individual gross income is the 
combination of all income earned in a year (e.g. wages, rental income, interest income, dividends, retirement distributions, etc.) 
AGI accounts for allowable deductions from the gross income. These deductions are subject change each year and can include, 
for example, pre-tax dependent care deductions and pre-tax retirement contributions. 

28 M. S. Chu and M. M. Kovalick, "An Analysis of the Free File Program," in National Tax Associate Proceedings, 99th Annual 
Conference on Taxation, Boston, 2006 

29 Internal Revenue Service and Free File Alliance, LLC, “Free On-Line Electronic Filing Agreement”, Internal Revenue Service, 
2002 . 

30 J. G. Russell, "2006 Tax Return Filing Season and the 2007 Budget Proposal for the Internal Revenue Service: Hearing before 
the U.S. House Of Representatives Committee on Ways and Means Subcommittee on Oversight (Statement of J. Russell 
George)," Department of Treasury, Washington D.C., 2006. 

31 Internal Revenue Service and Free File, Incorporated, "Memorandum of Understanding on Service Standards and Disputes," 
Internal Revenue Service, 2005 

32 The volume of eligible taxpayers has changed each filing year based on the most current AGI number that equates 70 percent 
of all individual income taxpayers. 
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one Free File offer, so while offers may vary across members, there must be full coverage by the 
Alliance as a whole. 

Table 1 summarizes current eligibility criteria for taxpayers by Alliance members as of August 
2019. MITRE discussions with members during site visit interviews helped illustrate—at a high- 
level—why companies chose to structure their eligibility criteria. Given that the 10/50 rule limits 
the percentage of the eligible population for each member, member companies make choices 
about how to tailor their program based on their own business models and any other factors not 
discussed. Members can use the “levers” of AGI, age, state, etc. to make adjustments so that they 
stay within the 10/50 rule bounds. As discussed in more detail in Section 4.3.3, the IRS provides 
members with a tool (spreadsheet) they can use to make adjustments year to year. 

Table 1. Taxpayer Eligibility Criteria by Free File Alliance Member (as of August 2019) 33 . 


Free File Alliance 
Member Company 

AGI 

Age 

Eligibility 
based on 
EITC 
eligibility? 

Eligibility 
dependent 
on state? 

Same 

eligibility 

criteria 

applies when 

filing with a 

foreign 

address? 

Free for 
active 
military 
for AGI of 
$66,000 or 
less? 

Free for state 

return if 
eligible for 
free federal 
return? 

1040NOW.NET 

$66,000 
or less 

Up to 60 
years** 

No 

Yes 

Yes 

Yes 

No free state 
returns 

eSmart Free File 

Edition 

$66,000 
or less 

Up to 53 
years 

No 

No 

No 

Yes 

No free state 
returns 

ezTaxRetum.com 

$66,000 
or less 

Any age 

No 

Yes 

No 

No 

No free state 
returns 

F ile Y ourTaxes .com 

$9,000 - 
$66,000 

Up to 65 
years 

No 

No 

Yes 

Yes 

Yes, for some 
states 

FreeTax Returns.com 

$66,000 
or less 

Up to 70 
years 

No 

Yes 

No 

Yes 

Yes, for some 
states 

FreeTaxUSAOIRS 

Free File Edition 

$35,000 
or less 

Any age 

Yes 

No 

No 

Yes 

Yes, for some 
states 

H&R Block's Free File 

$66,000 
or less 

17 years - 
51 years 

Yes 

No 

Yes 

Yes 

Yes, for all 
states 

Online Taxes at 
OLT.com 

$14,000 - 
$66,000 

Any age 

No 

No 

Yes 

Yes 

Yes, for all 
states 

TaxAct®Free File 

$55,000 
or less 

Up to 56 
years 

Yes 

No 

Yes 

Yes 

Yes, for all 
states 

TaxSlayer 

$66,000 
or less 

Up to 50 
years 

No 

No 

Yes 

No 

Yes, for some 
states 

TurboTax Free File 
Program 

$34,000 
or less 

Any age 

Yes 

No 

Yes 

Yes 

Yes, for all 
states 


** Applicable only for some states 


33 Internal Revenue Service, " Free File Software Offers," Internal Revenue Service, 30 August 2019. [Online]. Available: 
https://apps.irs.gov/app/freeFile/. [Accessed 30 August 2019], 
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These are the offers—and offer parameters—that taxpayers see on the IRS.gov Free File offers 
page. Overall, taxpayers utilizing Free File have the primary responsibility of identifying the 
right company based on their circumstance. As described in Section 1.1.1, the IRS offers a 
lookup tool to help taxpayers in identifying their eligibility across different member offers. 

1.1.3 Economic Drivers of Industry 

Though the Free File program has its roots in political directives, the design and implementation 
of the program is equally driven by economic forces. The MOU between the IRS and the 
Free File Alliance, which describes the terms and conditions for achieving the program’s 
objectives, is largely shaped by the economic incentives of the programs’ key stakeholders: 
taxpayers (consumers), the IRS (administrator), and tax preparation industry members (service 
providers). 

Consumer demand for Free File is expected to be a function of the “costs”—in time, money, and 
effort—of tax preparation and filing through Free File, and the availability of comparable 
substitutes. Similarly, the supply of free tax preparation services is a function of the economic 
returns that service providers expect to earn from their program investments. Because Alliance 
members are required to offer Free File services to eligible taxpayers free of charge, these 
companies must use other means to recover their costs and forgone revenue. 

The Free File partnership helps generate additional business for Alliance members, including 
from customers who end up using paid tax preparation services. Essentially, these commercial 
customers who pay for tax software of Alliance members help subsidize the provision of the free 
services. The IRS stands in a unique position of having to balance taxpayer expectations from the 
program with the industry members’ outlook for economic gains while ensuring that the agency 
remains cost-effective in implementing Free File. The MOU serves as a powerful economic 
negotiation tool that the IRS and Alliance members use to maximize the expected returns shaped 
by stakeholders’ economic incentives. 

These are critical factors in understanding industry motivations for participating in the 
partnership and providing free returns. Appendix A delves more deeply into the economic theory 
and forces driving the Free File program. 

1.1.4 Recent Interest in the Free File Program 

MITRE collected and reviewed several reports, studies, and articles as part of this 
assessment. These references included analysis of the Free File program by media, tax advocacy, 
and oversight organizations who proposed numerous recommendations for consideration. Rather 
than duplicate the efforts of these earlier studies, MITRE identified common themes and 
incorporated them into this assessment where applicable. The following key artifacts were 
reviewed by the MITRE team: 

• Internal Revenue Service Advisory Council (IRSAC). IRSAC is a federal advisory 
committee whose purpose is to serve as an advisor to the Commissioner of the Internal 
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Revenue. In 2018, the IRS requested that IRSAC evaluate the value of the Free File 
program and identify needed improvements 34 . 

The IRSAC assessment concluded that Free File is still a viable program worth IRS 
resources and sponsorship. However, the committee assessment also concluded that the 
IRS did not provide adequate oversight over the Free File program and that short- and 
long-term goals, objectives, and perfonnance measures needed to be clearly defined to 
more effectively assess program perfonnance and success moving forward. 

• Taxpayer Advocate Service (TAS). The National Taxpayer Advocate heads an 
independent organization within the IRS charged with ensuring that taxpayers are treated 
fairly and that they understand their rights. In February 2018, TAS issued a report to 
Congress that outlines several concerns regarding the IRS Free File program 35 . 

While the TAS report acknowledged that the IRS renewed its agreement with the Alliance 
multiple times to bolster taxpayer eligibility, strengthen security and privacy requirements, 
and require members to provide an electronic Free File indictor, it outlined several areas 
that the program fell short including insufficient oversight and evaluation, declining usage 
of the program, and a lack of a robust demographics analysis to detennine the cause, 
inconsistency in content quality across member companies, and cross-marketing and 
advertising of other services on Free File software platforms, which sometimes confuses 
taxpayers. 

• Treasury Inspector General for Tax Administration (TIGTA). TIGTA conducted a 
review of the Free File program in 2007 to detennine the IRS’ effectiveness in 
administering the program 36 . At the time of MITRE’s assessment, TIGTA was conducting 
an updated review of the Free File program, which is due to be published in Fall 2019. As 
such, this report did not consider TIGTA’s most recent findings and recommendations. 

• ProPublica Articles. ProPublica published a series of articles over the past several years 
on this subject, focusing primarily on the two largest tax software/preparation companies 
(the most relevant to this assessment published between March 20, 2017, and June 5, 
2019). ProPublica describes itself as a non-profit newsroom whose mission is “To expose 
abuses of power and betrayals of the public trust by government, business, and other 
institutions, using the moral force of investigative journalism to spur reform through the 
sustained spotlighting of wrongdoing.” 37 The publication alleged that the companies had a 
deliberate strategy to steer customers away from the free product and into paid versions in 
deceptive ways. They further contend the companies hid their free offerings from the 
Google search engine, and certain tax preparation companies lied to some customers who 
sought refunds in the wake of the reporting. See the section below for additional 
information on this series 38 . 


34 Internal Revenue Service, "Internal Revenue Service Advisory Council Public Report, Publication 5316, Catalog Number 
71824A," Internal Revenue Service, Washington, D.C., 2018. 

35 Taxpayer Advocate Service, "National Taxpayer Advocate Annual Report to Congress: Volume 1," Taxpayer Advocate 
Service, Washington, D.C., 2018 

36 Treasury Inspector General for Tax Administration, "Additional Action Is Needed to Expand the Use and Improve the 
Administration of the Free File Program," U.S. Department of Treasury, Washington, D.C., 2007 

37 ProPublica, "About Us," ProPublica, 2019. [Online]. Available: https://www.propublica.org/about/. [Accessed 2 July 2019]. 
38 Series of ProPublica articles from March 20, 2017 to June 5, 2019, available at https://www.propublica.org 
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A more detailed listing of findings and recommendations from external oversight organizations 
and stakeholders is in Appendix B, along with a crosswalk to MITRE’s own recommendations. 
An annotated list of ProPublica articles is in Appendix C. 

Political Interest: Role of Government and Public Sector in Software Preparation and Filing 

Underlying the recent focus on the Free File program is a divergence of views over whether the 
IRS should provide its own offering for taxpayers to electronically file their tax returns, or 
whether that is a function strictly for the private sector to perfonn as they have for more than 30 
years. 

A provision in the recently passed Taxpayer First Act that would have legislatively mandated the 
Free File program was taken out just before the Act passed. Many believed this provision would 
have barred the IRS from ever providing its own offering. In June, a group of nearly 40 advocacy 
and community groups, along with 75 academicians and tax experts sent a letter to Congress 
urging them to pull the provision and conduct an inquiry into the Free File program. The letter 
cited the series of articles by ProPublica described in the previous section. 

ProPublica reports that certain tax preparation software companies have lobbied for years to 
permanently bar the IRS from offering its own free alternative, arguing that governmental 
encroachment at both the federal and state levels may present a continued competitive threat to 
the industry for the foreseeable future. They say that the agreement with the companies not only 
keeps the federal agency from developing a direct filling option, but also an automated “return- 
free” option in which the IRS could calculate most individuals’ taxes. 

In 2010, MITRE conducted a feasibility study of the IRS offering its own e-filing program and 
concluded it was neither cost-beneficial, nor could the IRS keep pace with the innovation of the 
private sector 39 . MITRE recognizes that the environment of government services provided to its 
citizens is continually evolving, and that Congress may determine in the future whether the IRS 
should offer that service and provide it with the appropriate funding to do so. 

MITRE was not asked to weigh in on this debate with this assessment, but rather to look at the 
Free File program itself in terms of the member compliance with the MOU and to make 
recommendations for the program based on its assessment. However, it is essential to understand 
the underlying political arguments, because simply knowing whether a company is in 
compliance with an MOU that was jointly negotiated between government and industry may not 
satisfy those who want the program investigated. 

In this report, MITRE considered all sides of the program, as evidenced by the broad scope of 
the assessment outlined below. 

1.2 Scope 

As part of this independent assessment, MITRE conducted a comprehensive review of the 
current IRS Free File program from three perspectives: Free File Alliance and its member 
organizations, IRS Free File Program Office leadership and staff, and taxpayers. 

Specifically, the scope of the assessment includes: 


39 The MITRE Corporation, "Advancing E-file Study - Phase 2 Report," Internal Revenue Service, Washington, D.C., 2010. 
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• The multi-year agreement/MOU structure that governs the Free File program, and how it 
could be improved 

• Member compliance with specific provisions of the current MOU 

• Compliance and oversight activities conducted by the IRS Free File Program Office 

• Stakeholder understanding of historical and current objectives of the program and vision 
of success 

• Assertions from media (specifically ProPublica) and recommendations from oversight 
organizations IRSAC, TAS, and TIGTA 

• Taxpayer experience with the Free File program, based both on an assessment by human 
factor and user-experience engineers and usability testing with actual taxpayers 

• Observations and suggestions from the Alliance, IRS, taxpayers, and other stakeholders 
for improvements to the program 

• Web forensics analysis with focus on search engine coding and search traffic 

• Data analysis of e-filed/Free Filed returns in the ecosystem of return filings 

• Analysis of the PPP and the role it plays in the Free File program 

• Economic analysis of the Free File program 

Areas that were out of scope for the assessment included: 

• State tax filing. The assessment focused on the federal Free File program only. While 
there are aspects of the program that pertain to states, MITRE did not consider state tax 
filing itself, nor did MITRE speak with any state stakeholders. 

• Interviews with oversight agencies or media organizations. While MITRE reviewed 
relevant documentation from oversight agencies and articles in the press (specifically 
ProPublica), MITRE did not conduct interviews or speak directly with either. 

• Free File Fillable Forms. Taxpayers with no restriction on AGI can opt to fill out 
electronic versions of IRS fonns and submit electronically for free from IRS.gov. While 
Free File Fillable Forms (FFFF) are provided to the IRS by the FFA, the form is 
unbranded. MITRE considered FFFF only in terms of overall usage of the Free File 
program and as part of responsibilities of IRS Free File Program Office. Beyond that, 

FFFF is not part of this assessment. 

1.3 Assumptions and Limitations 

MITRE conducted this assessment with the following assumptions: 

• As discussed in Section 1.1.4, for the purposes of this study, MITRE assumes that industry 
will continue, at least in the near future, to be the entity that provides free return offerings 
to taxpayers. All of MITRE’s recommendations in this report follow that assumption. 

• Keyword search in our web analysis is based on a representative sample of organic 
(unpaid) search keywords for free tax filing, via two primary search engines, Google and 
Bing. There are many other search engines that taxpayers could use (Yahoo, Baidu, ASK, 
AOL, etc.), though Google and Bing are the largest. While we used more than 100 of the 
top search terms, the terms were not exhaustive. 
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• MITRE used TY 2017 data from the IRS CDW in the data analysis portions of the 
assessment. Though TY 2018 data is available, because taxpayers with extensions can file 
2018 tax returns until October 15, 2019, it is not considered a full data set. Therefore, 
MITRE used TY 2017 CDW data for all analysis, unless otherwise stated. 

• MITRE conducted an initial assessment that included: a landing page compliance 
assessment, a search avoidance analysis, and a legal analysis. The results are included in 
this report where appropriate, and results in this report supersede any results shared with 
the IRS before this formal deliverable. 

In addition, to preserve anonymity, MITRE “masked” member company names throughout this 
report—with the exception of the FS 2019 member offerings, which are available publicly on 
IRS.gov. MITRE provided the IRS with a member key. 

The MITRE team also operated under several limitations and constraints for this study, 
including: 

• Data limitations. 

o All values in Figure ES-2 and Figure 7 were drawn from CDW queries for TY 2017 
(FS 2018) with the exception of the fillable form portion of the Free File number. 
Because fillable fonns were not in the query, MITRE used data provided from the IRS 
(W&I Research, Electronic Tax Administration Research and Analysis System 
[ETARAS]) of 295,000 fillable fonn users. The Free File number differs slightly as 
the IRS uses calendar year data. 

o The number of free commercial filings cited were provided by the FFA by each of the 
current members (one member recently dropped out of the program and did not 
provide numbers). These numbers are not publicly available, and MITRE cannot 
independently verify them. 

o Absence of detailed industry, firm, and taxpayer-level data on electronic tax 
preparation and filing services it is difficult to develop a full economic 
characterization of Free File. Notwithstanding, the economic analysis in Appendix A 
uses available proprietary and public data, an extensive literature review, and simple 
economic concepts to establish an economic perspective of the program. 

• FFA member interviews. Ten of the 12 interviews with members were in person. One 
was by phone with written responses provided, and one was just by written response. 

• Proprietary information. In general, members were very candid and open. As would be 
expected, their legal counsel advised them to not answer questions that were deemed to be 
proprietary. However, that was not a limitation in terms of determining compliance with 
the MOU and MITRE was able to complete our assessment for each member. 

• Self-attestation. MITRE was not able to physically observe compliance with some 
provisions of the MOU. For those provisions, the members either verbally or in writing 
attested to their compliance. 

• Taxpayer journey limitations. For taxpayer journey activities, the MITRE team created 
user accounts to access each of the member software packages. In doing so, the team was 
able to simulate tax preparation for three scenarios for each package. However, the team 
did not actually file taxes using these accounts. Therefore, any screenshots or activities 
that would take place after submitting the return were out of scope for the team. 
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• Usability specific limitations. The taxpayer usability study has several limitations that 
may affect the external validity of this study, meaning the extent to which the results of 
this study can be generalized beyond the context of the study setting. The complete set of 
limitations are in the accompanying Taxpayer Experience Results report, Section 1.3. 

1.4 Organization of This Document 

This document is organized into the following sections, as shown in Table 2. 

Table 2. Document Organization. 


Section Number and Title 

Contents 

1. Introduction 

Describes the reason for this assessment 
and provides relevant background 
information to set context. Outlines the 
scope of the assessment, any assumption 
and limitations, and the organization of this 
document. 

2. Methods Overview 

Provides a high-level overview of the 
methodological approach to this study. 

Findings 

3. Eligible Taxpayers and Actual 

Provides findings from TY 2017 data analysis 

Participation 

of eligible taxpayers 

4. Compliance with the MOU 

Provides overview of current MOU, 
compliance scorecard for all FFA members, 
discussion of search and search avoidance, 
results of Program Office oversight activity 
review, and guidance for assessing public- 
private partnerships 

5. Taxpayer Experience with Free File 

Provides detailed findings from expert 
usability review (taxpayer journey) and 
taxpayer usability testing 

Recommendations and Appendices 

6. Recommendations 

Provides consolidated findings and 
recommendations for program office, MOU 
updates based on program assessment, 

MOU updates based on legal assessment, 
taxpayer journey and taxpayer experience 

7. Appendices 

Economics of Free File, crosswalk of MITRE 
recommendations and oversight reports, 
taxpayer journey results, taxpayer usability 
results, search engines overview, etc. 
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2.0 Methods Overview 


For this comprehensive assessment of the Free File program, MITRE used a cross- 
disciplinary approach to focus on three key objectives: assessing Free File program 
management and oversight, assessing FFA member compliance, and assessing overall 
taxpayer experience. Highlights of this approach include: 

• Multiple workstreams worked concurrently to consider the program from 
different perspectives. This included a data analysis of the eligible population; a 
comprehensive web analysis that included web forensics to determine where taxpayers 
landed from organic searches and how coding and paid advertising influenced that; 
taxpayer usability from actual taxpayers’ experience as well as human factor engineers 
conducting heuristic analyses of both IRS.gov and member sites; and a legal analysis of 
the MOU to highlight changes that could improve the program. 

• Taken together, this allowed MITRE to take a holistic approach to assessing the 
Free File program. While these interdisciplinary workstreams worked independently, 
they each worked toward the same stated objectives to bring together a holistic view of 
MOU compliance and improvements that could be made to the program. 

• Each workstream used rigorous methods to produce findings contained in this 
report. Each discipline used the appropriate methodology to apply rigor and clearly 
document a data-driven, evidence-based approach to determine recommendations. 


Beginning in June 2019, MITRE established several workstreams for addressing a set of targeted 
assessment areas. Taken together, these workstreams produced findings and recommendations 
related to Free File member compliance with the MOU and potential enhancements to the 
Free File program. Highlights of the assessment approach include: 

• Current environmental scan. MITRE conducted an environmental scan to better 
understand the Free File program in terms of historical context, taxpayer participation, the 
role of the IRS and industry in its PPP, the economics of “free” as it pertains to Free File, 
and recent interest in the program. 

• Free File program analysis. MITRE conducted stakeholder interviews—including with 
IRS leadership—and performed document reviews to assess the IRS Free File program’s 
alignment to objectives, performance, and the IRS management and oversight functions. 

• FFA member organization interviews (“site visits”). MITRE conducted interviews with 
representatives from each Free File member organization to collect information on their 
interactions and agreements with the IRS regarding the Free File program and applicable 
business processes, practices, and techniques for how each Free File member determines if 
a taxpayer qualifies for Free File. Each of the interviews followed a similar approach: 
standard interview protocols, targeted questions (both related to the IRS Free File program 
as well as the MOU Article 4, Standards of Practice) and in-person or phone interviews 
with participants from each Free File member organizations (selected by the member 
company’s leadership). 
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• Web forensics analyses. MITRE performed an analysis to detennine if any coding within 
the member’s website influences search engine routing. In addition, MITRE analyzed web 
search query traffic metrics for internet searches that signaled an intent to find free tax 
filing services, based on a representative sample of organic (unpaid) search keywords for 
free tax filing. This included a click-through analysis ofIRS.gov site usage metrics to 
investigate where taxpayers went from the IRS Free File portal web page. This analysis 
was also used to determine how often taxpayers searched for free tax filing services 
(especially during tax filing season) using two primary search engines, Google and Bing; 
to what site(s) these users went following their searches; and the differences in search 
engine results. 

• Taxpayer journey. MITRE conducted an expert review of the IRS Free File portal as 
well as each individual Free File member offer. This review was twofold in its objective: 
first, MITRE evaluated the 12 member sites for compliance with MOU requirements 
(including an initial landing page analysis); second, MITRE evaluated the 12 member sites 
and the Free File-related pages on IRS.gov against recognized software user interface 
usability principles (called heuristics) in order to identify potential usability problem 
areas. For both parts, MITRE set-up user accounts and ran scenarios to test the sites in a 
systematic, exhaustive fashion. MITRE’s usability assessment used both industry- 
recognized criteria (heuristics) as well as MITRE’s interpretation of how MOU 
requirements translate into usability criteria and whether sites are optimized for mobile 
devices. 

• Taxpayer usability testing. MITRE conducted a study to test the usability of the IRS 
Free File portal and assess taxpayers’ actual experience in filing tax returns using three 
randomly selected Free File software offerings. MITRE conducted in-person usability 
testing sessions with 29 taxpayers who filed their taxes electronically using Free File 
software in prior year filings. During the testing, taxpayers were observed as they 
completed fictitious tax scenarios using the IRS Free File portal and Free File software. In 
addition to data on the usability of the Free File services, MITRE collected and analyzed 
data on the frequency of errors committed, taxpayer satisfaction, and experience (via a 
brief questionnaire), and taxpayers’ perceptions of the Free File program. 

• Legal analysis. MITRE’s General Counsel, in coordination with outside counsel, 
conducted a review of the MOU’s terms and conditions to detennine recommended 
revisions to the MOU to more clearly define the parties’ responsibilities to enhance and 
promote the IRS Free File program, address any shortcomings in the MOU language as it 
relates to the prohibition against unfair methods of competition, and provide enforcement 
capability via liability clauses if the MOU's terms and conditions are breached by a 
Free File member organization. 

• Recommendations. Based on the data gathered from the workstreams, MITRE developed 
recommendations for overall improvements to the Free File program, potential 
enhancements to the MOU, and improvements to usability ofIRS.gov and some aspects of 
member offerings. 

Figure 2 shows how each of these workstreams collectively contributed to addressing targeted 
assessment areas (assessment objectives) including: documenting the current Free File program 
environment, assessing Free File program management and oversight, assessing FFA member 
compliance with the MOU, assessing taxpayer experience with respect to the program, and 
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ultimately drafting recommendations for the program and MOU. Outputs from some 
workstreams contributed to more than one assessment area. 



Document Current 
Free File Environment 



V Economics 
Z Filers 

V Trends 

•f Free File Market 
'f External Literature 


\ 


Assess Free File Program 
Management & Oversight 


■S Program Objectives 
Z Program Management 
Z IRS Compliance Approach 
V FFA Compliance Approach 





Assess Free File Alliance (FFA) 
Member Compliance 


Taxpayer Experience 



S IRS Data Analysis 

V Web Analysis 

V Taxpayer Journey 
Z FFA Leadership and 

Member Interviews 


V Taxpayer Journey 
Z Taxpayer Usability Testing 


Develop Program & MOU 
Recommendations 


v' Legal Review 
Z SME Review 
v' IRS Interviews 
•S FFA Leadership and 
Member Interviews 
•S Workstream Findings 


Figure 2. High-Level Approach to Free File Program Assessment (Aligned to Assessment Objectives). 

2.1 Current Environmental Scan 

As part of understanding the overall Free File environment, MITRE conducted a review and 
analysis of IRS-provided program documentation, published reports from external stakeholder 
groups (i.e., IRSAC, TAS, TIGTA) regarding the Free File program, and recent media 
publications. 

Public-Private Partnership 

Research shows that while collaboration across organizational and sectoral lines is both 
necessary and desirable to address complex challenges, collaboration is neither easy nor always 
effective. One particular challenge is that government practitioners increasingly face demands 
for concrete and tangible ‘results’—i.e., the visible creation of public value. 40 


40 S. Goldsmith and W. D. Eggers, Governing by Network: The New Shape of the Public Sector, Washington, D.C.: The 
Brookings Institution, 2004. 
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Performance and costs, however, are not the only criteria for assessing the value partnerships 
create. Research suggests using a broader set of criteria for assessing whether partnerships are 
successful. For example, other key attributes of successful PPPs including democratic 
responsiveness and process legitimacy, are considered to be equally important values for public 
undertakings. 41 

As part of this assessment, MITRE therefore conducted a review of the Free File PPP through 
the perspective of alterative attributes to help evaluate the value of the program. In addition to 
interviews with each partner and review of literature and documentation regarding the Free File 
program, MITRE applied a comprehensive framework for assessment of cross-sector 
collaborations 42 to assess the Free File program PPP. 

Using indicators of democratic responsiveness, process legitimacy, and substantive outcomes as 
the attributes of value, MITRE mapped specific Free File indicators to holistically review the 
partnership. 

Economics of Free File 

MITRE conducted an economic analysis to determine the fundamental links between economic 
theory and the Free File program. The analysis started with an environmental scan of the 
program and its key stakeholders and looked at taxpayers’ demand for Free File as a function of 
the economic price (cost) of the program relative to other tax preparation and filing methods. A 
demographic analysis of taxpayers was also performed. The analysis explored the economic 
motivations of the IRS to partner with private sector companies in providing Free File. 
Subsequently, the analysis addressed the software industry’s market structure and economic 
incentives for offering free tax preparation and filing services. 

MITRE also looked at the MOU as a two-way tool for economic negotiations between the IRS 
and the Alliance that determines the dynamics of demand and supply of Free File services. 
Subsequently, the analysis evaluated the economic explanation relative to MITRE’s findings on 
IRS’s oversight of the program as well as the Alliance members’ compliance with the existing 
MOU. The complete economics report is in Appendix A. 

2.2 Free File Program Analysis 

MITRE conducted an outcome and process evaluation of the IRS Free File program, guided by 
the following research questions: 

• Is the IRS Free File program achieving the IRS’s desired objectives? 

• Are program resources being used efficiently? 

• Are the IRS’s oversight and compliance processes allowing the IRS to effectively manage 
the provisions of the MOU? 

To obtain an understanding of the current state of the management of the Free File program, 
MITRE conducted interviews with IRS program personnel, IRS leadership, and Alliance 
leadership. These interviews supplemented MITRE’s review and analysis of IRS-provided 


41 R. D. Behn, Rethinking Democratic Accountability, Washington, D.C.: The Brookings Institution, 2001. 

42 S. B. Page, M. M. Stone, J. Bryson and B. C. Crosby, "Public value creation by cross-sector collaborations: A framework and 
challenges of assessment," Public Administration, vol. 93, no. 3, pp. 715-732, 2015. 
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program documentation, published reports from external stakeholder groups (i.e., IRSAC, TAS, 
TIGTA) regarding the Free File program, and recent media publications. 

Stakeholder interviews were focused on gaining a full and accurate understanding of the current 
management and operational oversight of the Free File program and MOU compliance by the 
IRS as the program sponsor and through its collaboration with the Alliance. A list of stakeholder 
interview participants is provided in Table 3. 

Table 3. Stakeholder Interview Participants. 


Organization 

Title 

IRS Free File Program 

Office 

• Free File Program Director (Chief, Industry Engagement 

and Strategy) 

• Free File Program Team Lead 

IRS Leadership 

• Deputy Director, Submission Processing 

• Director, e-file Services, Submission Processing 

IRS W&l Customer 

Account Services (CAS) 
Electronic Products 
Support & Services (EPSS) 

• Chief, EPSS 

• Chief, EPSS Operations Support 

• e-File Provider Program Manager 

• Chief of e-Help Operations 

Free File Alliance 
Leadership and 

Contractor 

• FFA Executive Director 

• FFA Legal Counsel 

• FFA Independent Compliance Auditor 


2.3 Free File Alliance Member Organization Interviews (Site Visits) 

MITRE conduced fact-finding interviews with the 12 Free File member companies to assess 
alignment with the MOU, perceived value of the program, and any recommendations for 
enhancements to the program. The objectives of the interviews were used to inform the 
following aspects of the Free File program assessment: 

• FFA approach to monitoring member compliance to the MOU 

• Member companies’ compliance practices 

• Recommendations from FFA member companies for enhancements to the Free File 
program 

• Suggestions for clarification or improvements to the MOU 

MITRE developed an interview questionnaire and conducted phone interviews with FFA 
leadership and face-to-face interviews with representatives of 10 of the 12 Free File member 
organizations. One interview was conducted by phone with written responses provided, and one 
was handled via written correspondence. The member questionnaire contained targeted questions 
related to the IRS Free File program as well as the MOU Article 4, Standards of Practice to 
determine Free File member compliance with the current Free File MOU and solicit views of the 
overall Free File program. 

These interviews allowed MITRE to gain an understanding of member interactions and 
agreements with the IRS Free File program and applicable business processes, practices, and 
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techniques for how each Free File member monitors compliance and determines if a taxpayer 
qualifies, as well as the procedures or practices to determine when taxpayers do not qualify for 
Free File. 

2.4 Web Forensics Analyses 

MITRE conducted multi-tiered web analyses, including data analysis on web search query traffic 
and an analysis of organic and paid web search traffic flow from Google to the Free File member 
landing pages and related free tax filing web pages during the FS 2019. 

As part of the initial assessment, MITRE captured and reviewed the source code associated with 
each Free File program member’s landing page to determine if any coding within the member’s 
page violates the terms of the MOU. 

MITRE used a variety of web analytics tools and techniques to answer the following research 
questions regarding usage metrics during the 2019 tax filing season (to the extent possible). For 
each question, the approach that was used to answer the question is shown below. 

1. For each partner in the FFA, to which URLs (member landing pages) were taxpayers 
directed from the Free File program portal page on the IRS.gov website? 

Approach: Followed each of the li nk s on the IRS.gov Free File program pages and 
observed any changes in URLs (caused by web page redirects - see also number 3 below). 

2. How much web traffic navigated from the Free File portal page on IRS.gov to the 12 
member landing pages? 

Approach: Used Google Analytics reporting to analyze the web traffic metrics. 

3. MITRE observed that some of the member li nk s were redirecting the taxpayer to a URL 
that was different than the URL shown on the IRS.gov Free File program page. Did these 
pages always redirect? If not, can we see any differences between the original URL and 
the new target page? 

Approach: MITRE used Wayback Machine® 43 to see if a copy of the old URLs existed in 
their database. If so, MITRE captured the differences in the two pages. 

4. Is there anything about the 12 member landing pages that would modify how these pages 
were handled by the major search engines? For example, do they instruct the search 
engine to exclude the member Free File landing page from the search index? 

Approach: Analyzed the robots.txt file for the presence of "disallow" statements that 
would prevent search engines from crawling the member landing pages. Also looked at 


43 Internet Archive, "Wayback Machine," Internet Archive, 31 December 2014. [Online]. Available: https://web.archive.org/. 
[Accessed 15 July 2019], 
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the source code for each of the member landing pages for presence of meta 
robots=noindex, checked for meta refresh redirects, and also rel=canonical tags. 

5. During FS 2019, how much web traffic with search queries such as “free tax filing” 
navigated from Google organic and paid search to the various FFA partner sites? Were the 
target URLs of this traffic the same or different than the FFA landing pages? Did any 
traffic go from Google to the FFA landing pages? 

Approach: MITRE used a leading third-party keyword research tool 44 to analyze ranking 
data of organic and paid search results for each of the members. 

6. How much money were FFA partners willing to spend on searchers who were looking for 
free tax filing services? 

Approach: MITRE used additional third-party data to analyze average cost per click of 
purchased search keywords and estimate cost of the projected traffic delivered via these 
paid search results. 

7. Once a Free File user reached the IRS.gov website, what percentage of people used the 
various paths to get to the IRS Free File program web page? 

Approach: MITRE used Google Analytics data to track click paths, next page flows, etc. 
MITRE included link navigation and site search. 

As part of the data analysis on web searches, MITRE captured query traffic metrics for users 
who searched with queries that signaled an intent to find free tax filing services. This also 
included a click-through analysis ofIRS.gov site usage metrics to investigate where taxpayers 
went from the IRS Free File web page. 

MITRE used this analysis to detennine: 

• How often taxpayers searched for free tax filing services (especially during tax filing 
season) 

• To what website(s) did these users go following their searches and (to the extent possible 
through available data) 

• To which Free File program member site(s) did taxpayers go after arriving at the IRS 
Free File page 

• Were there any differences between where taxpayers went from the IRS Free File page 
versus where they went from using a search engine such as Google 

In addition, MITRE downloaded the top 30,000 search terms that drove organic search traffic 
and the top 30,000 search tenns that drove paid search to each of the 12 Free File member sites 
as well as to the IRS.gov website. 


44 SEMrush Inc., "All-in-one Marketing Toolkit," SEMrush Inc., 2019. [Online]. Available: https://www.semrush.com/. 
[Accessed 15 July 2019], 
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For each search term that drove traffic to a specified domain, several attributes were captured. 
The most important for this effort are: 

• Search keyword (what people were searching for) 

• The URLs of the Free File member websites that ranked in search for each keyword 

• Search position of the ranking URL for each keyword 

• An estimated monthly search volume for each keyword on Google US (the 12-month 
average ending in May 2019 was seasonally adjusted for the filing season) 

• An estimated average monthly traffic driven to the ranking URL for each keyword 

• Cost per click for paid ads purchased for each keyword 

MITRE then flagged each URL in the data that matched any of the 12 Free File member site 
landing pages. 

From the data, MITRE could identify a representative sample of high value keywords that drove 
traffic to FFA landing pages (108 keywords) and matched following criteria: 

• Caused at least one FFA member page to rank in the top 10 search positions 

• The search was not a branded search term (i.e., did not contain a member company’s 
commercial product names) 

• The search did not contain a year designation within the keyword (e.g., 2016) 

• The search did not contain the word “state” (this eliminated the searches targeted for the 
free state tax filing products) 

MITRE also captured ranked searches for the non-free file landing pages. These were identified 
and included in the target dataset for the analysis. For example, a search of “free tax filing” 
caused a non-Free File Alliance landing page to rank in the second highest search result position. 

2.5 Taxpayer Journey 

MITRE performed multiple research activities as part of the taxpayer journey analysis. The 
objective of these research activities was twofold: assess member Free File offerings for 
compliance with MOU and assess usability of each offering (from a taxpayer’s perspective). For 
the first part, MITRE assessed the 12 FFA member companies’ compliance with MOU 
requirements by evaluating the member landing pages (as part of the initial assessment), 
followed by a more comprehensive review of member sites in which the MITRE team set-up 
user accounts and ran scenarios to test for observable MOU compliance. MITRE incorporated 
the assessment of observable compliance with these MOU requirements into overall 
detenninations of member compliance and noncompliance, as reported in Section 4.2. 

For both the compliance and usability reviews, MITRE human factors experts used fictitious 
taxpayer scenarios to navigate the user experience (UX). The human factors team used three 
scenarios to select a member offer, create an account with the member sites, and proceed up to 
the point of filing a return (returns were not actually filed for these exercises). The three 
scenarios were: 

1. Eligible taxpayer “simple” scenario. A single taxpayer with no dependents, who has only 
income reported on a Form W-2. The taxpayer is eligible for Free File. 
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2. Eligible taxpayer “complex scenario.” A married filing jointly taxpayer, with a spouse 
and two dependent children. The taxpayer reports income from two jobs (two Form W- 
2s) and interest income on Form 1099-INT. The taxpayer is eligible for Free File. 

3. Ineligible scenario. A single taxpayer who reports income from a Form W-2. The 
taxpayer is ineligible for Free File due to having wage income of $74,000. 

For the usability assessment, MITRE conducted an expert usability review of the 12 members 
sites in their entirety, as well as IRS.gov web pages related to the Free File program (including 
the Free File landing page, software offers page, and software lookup tool, as well as how 
taxpayers might access the Free File pages from web queries or other parts ofIRS.gov). To 
conduct this expert review, MITRE evaluated the sites against criteria known as heuristics. 
Heuristics represent widely recognized usability best practices. MITRE used Nielsen’s 10 
usability heuristics to assess user experience. 

Additionally, MITRE examined the MOU for requirements related to taxpayer experience and 
system usability and used these MOU requirements to derive a set of heuristics related to MOU 
adherence (UX MOU requirements). This set of MOU-related heuristics should not be 
interpreted as evidence of member compliance or noncompliance with the MOU. Finally, 

MITRE evaluated the extent to which member sites are optimized for mobile devices. 

In summary, MITRE’s heuristic review conducted under the taxpayer journey analysis evaluated 
member sites and IRS.gov using the following criteria: 

1. Nielsen’s 10 usability heuristics 45 . These 10 criteria—heuristics—developed in 1994 by 
Jakob Nielsen are widely recognized measures of system usability. These general 
principles for interaction design, as shown in Figure 3, are well referenced criteria for 
describing usable interfaces within the user experience community. Taken together, the 
10 heuristics construct a measure of user experience. They include factors such as “match 
between system and the real world,” “error prevention,” “flexibility and ease of use,” and 
“consistency and standards.” 

2. MOU requirements from a usability perspective (UX MOU adherence). MITRE 
examined MOU requirements pertaining to the user experience and constructed usability 
criteria from these MOU requirements. The usability assessment looked at the users’ 
experience (or expected experience) with the aspect of the website governed by the 
requirement, or how the MOU requirements are demonstrated via the user experience. 
These criteria included factors such as “understands offerings available” and 
“transparency around ineligibility.” MITRE’s assessment against these criteria are not 
included in the evaluation of MOU compliance presented in Section 4.2. For the member 
site assessments, MITRE also reviewed specific MOU requirements and assessed 
alignment to each from a usability perspective. Note that this is different than the 
assessment in Section 4.2, which assessed whether or not the member was in compliance 
with the MOU requirement. 


45 Nielsen’s 10 Usability Heuristics: Nielsen, Jakob. (1994). Heuristic evaluation. In Nielsen, J., and Mack, R.L. (Eds.), Usability 
Inspection Methods, John Wiley & Sons, New York, NY. 
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3. Mobile experience. MITRE evaluated the extent to which the examined sites were 
optimized for mobile device use. 

Figure 3 displays Nielsen’s 10 heuristics used for the user experience portion of the heuristic 
review. Figure 4 displays the heuristics MITRE constructed to evaluate whether the user 
experience aligns with MOU requirements. 


Nielsen’s 10 Usability Heuristics 

Visibility of system 
status 

The system should always keep users informed about what is going on, 
through appropriate feedback within reasonable time 

Match between 
system and the real 
world 

The system should speak the users' language, with words, phrases and 
concepts familiarto the user, rather than system-oriented terms. Follow 
real-world conventions, making information appear in a natural and 
logical order 

User control and 
freedom 

Users often choose system functions by mistake and will need a clearly 
marked "emergency exit" to leave the unwanted state without having to 
go through an extended dialogue. Support undo and redo 

Consistency and 
standards 

Users should not have to wonder whether different words, situations, or 
actions mean the same thing. Follow platform conventions 

Error prevention 

Even better than good error messages is a careful design which prevents a 
problem from occurring in the first place. Either eliminate error-prone 
conditions or check for them and present users with a confirmation 
option before they commit to the action 

Recognition rather 
than recall 

Minimize the user's memory load by making objects, actions, and options 
visible. The user should not have to remember information from one part 
of the dialogueto another. Instructions for use ofthe system should be 
visible or easily retrievable whenever appropriate 

Flexibility and 
efficiency of use 

Accelerators -- unseen by the novice user - may often speed up the 
interaction for the expert user such that the system can cater to both 
inexperienced and experienced users. Allow users to tailorfrequent 
actions 

Aesthetic and 
minimalist design 

Dialogues should not contain information which is irrelevant or rarely 
needed. Every extra unit of information in a dialogue competes with the 
relevant units of information and diminishes their relative visibility 

Hep users recognize, 
diagnose, and 
recover from errors 

Error messages should be expressed in plain language (no codes), 
precisely indicate the problem, and constructively suggest a solution 

Help and 
documentation 

Even though it is better if the system can be used without documentation, 
it may be necessary to provide help and documentation. Any such 
information should be easy to search, focused on the user's task, list 
concrete steps to be carried out, and not be too large 


Figure 3. Nielsen's 10 Usability Heuristics. 
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UX MOU Requirements 

Privacy, security, free 
from marketing 

User recognizes the system is secure and their privacy is protected. The 
system does not link to its paid offerings, nor does it request any 
unnecessary personal information, such as billing information (except 
when legitimately offering a state return for a fee). (Reference: 4.1.3vi; 
4.32.6; 4.19.3; 4.19.4) 

Understands offering 
available 

User feels confident the forms and schedules they use are supported by 
the system, and understands which forms or schedules are not 
accepted. They see whether or not they can file their state return for 
free. If free filing is not offered for their state, they understand there is a 
fee and see the cost. They see a prominent link back to the IRS.gov Free 

File Landing Page on the systems’ landing page. Prior to and during their 
experience, the user has a clear sense of their free customer service 
options. (Reference: 4.5.3; 4.5.4; 4.21.1; 4.32.1; 4.15.4; 4.32.7) 

User control and 

access 

A user has the option to complete their return at a later date. They may 
choose to print their return forfree and is aware of the option to do so. 

User may see the status of their return and obtain a copy after electronic 
filing. (Reference: 4.15.4; 4.15.4; 4.15.3; 4.15.1) 

Meets expectations 

User links to expected free file site from IRS.gov. The system meets 
user's expectations for the name of the site. User understands that the 
system is a legitimate Free File site via an appropriate logo. (Reference: 
4.19.3; 4.33; 4.34) 

New technologies 

Users are offered new technologies that pre-populate forms, such as 
uploading a W-2, when available. (Reference: 4.36.2) 

Transparency around 
ineligibility 

A user understands that they are ineligible as early as possible in the 
process. They understand why they are ineligible when informed. The 
user clearly sees a link back to IRS Free File Landing Page so that they 
may explore other options. In addition to understanding they may return 
to IRS.gov, the user may see that they may complete their return for a 
fee. In addition, an ineligible user who attempts to use the system if 
offered the option to print out their return for a fee equivalent to the 
systems commercial offerings. (Reference: 4.15.2; 4.19.2 vi;4.19.2 1; 

4.19.2 ii; 4.19.2 iii; 4.19.2 iv; 4.32.2; 4.32.5) 


Figure 4. Heuristics to Evaluate MOU Adherence from a Usability Perspective. 

For all three of these groups of criteria, the objective of the evaluation was to identify potential 
usability issues and prioritize them via a 1-5 (1 = worst, 5 = best) severity score. When multiple 
issues were found against a heuristic, the lowest score was used in calculating the overall score. 
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2.6 Taxpayer Usability Testing 

MITRE conducted in-person usability testing sessions with taxpayers who were identified in 
CDW as having submitted a tax return with the Free File program in TY 2018. MITRE sent 
invitation letters to 2,500 randomly selected prior Free Filers in Chicago, IL, and scheduled 
interviews with taxpayers who responded to the invitation. In total, MITRE completed 29 
usability testing sessions with Chicago taxpayers. 

The objectives of these interviews were to: 

• Understand how taxpayers locate the Free File program options; understand how 
taxpayers select a software provider 

• Capture usability issues with IRS websites related to the Free File program 

• Capture usability issues with specific vendor platfonns 

• Understand taxpayers’ previous experiences with the Free File program 

During the 90-minute sessions, participants completed fictitious scenarios to evaluate the 
experiences of completing three tasks: 1) locating the Free File program online; 2) selecting a 
Free File software offer, including with the software lookup tool; and 3) completing a tax return. 

MITRE randomly selected three Free File member software offers to test with taxpayers 46 ; each 
participant filed a fictitious return with one of the three selected offers. Taxpayers used mocked- 
up infonnation (e.g., fictitious Fonn W-2s and other documentation) to complete the tasks. 
MITRE researchers observed the session and then asked debriefing questions to establish what 
aspects of the Free File system (both the IRS.gov Free File pages and the specific vendor pages) 
caused confusion, errors, or frustration. In addition to data on the usability of the Free File 
system, MITRE also collected and analyzed data on taxpayer satisfaction and experience (via 
brief questionnaires), and taxpayers’ perceptions of the Free File system components and the 
Free File program. 

To complete this study, MITRE performed the following activities: 

1. Identified candidates for participation. The total universe of Free Filers in TY 2018 
was 2,343,235 47 . To select a location to conduct interviews, MITRE reviewed the volume 
of TY 2018 Free Filers in four metropolitan areas 48 to determine whether the volume was 


46 Due to limitations of time, MITRE was only able to test a sample population of 25-30 taxpayers, which limited usability testing 
to three Free File member software systems. The three members were selected randomly according to a methodology outlined in 
more detail in Appendix G, Taxpayer Experience Testing Report. 

47 These data were drawn from CDW (ETARASMEF1544 table), with the Free File indicator 

(ENGLISH FREE FILE RET_IND=1), accepted return indicator (accepted_rtn_ind=l), and a non-missing software vendor ID 
number (software_id number not null). The data were cleaned to remove duplicate taxpayer identification numbers, records with 
international zip codes (zip=0), records with missing zip code, and taxpayers under age 18. 

48 MITRE reviewed Free Filer density and characteristics in four clusters of zip codes. The clusters were in Chicago, IL; 

Portland, OR; Pittsburgh, PA area; and Dallas, TX area. MITRE previously visited these four sites to conduct a usability study of 
the 2018 redesigned 1040 fonn, and was therefore already familiar with the locations. In that study, MITRE interviewed 
taxpayers who filed on paper (a population that does not overlap with Free Filers). The clusters were formed for the F1040 study 
because they were geographically small enough to allow MITRE to interview taxpayers from a single interview venue, yet 
densely populated enough to yield sufficient taxpayers to send invitation letters. See the Taxpayer Experience Testing Report for 
a more detailed review of the sampling procedure and a comparison of the characteristics of the full U.S. Free Filer population 
with the sampled Chicago zip codes. 
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enough to send invitation letters and derive an interview sample. MITRE identified 
Chicago, IL as the best candidate location for the taxpayer interviews. The Chicago, IL 
cluster included 4,106 taxpayers who used Free File in TY 2018 (according to CDW 
indicators). From this pool of 4,106, MITRE randomly sampled 2,500 taxpayers to 
receive invitation letters. (See the Taxpayer Experience Testing Report appendices for 
copies of all recruitment materials.) 

2. Recruitment. Of the 2,500 invitation letters mailed, MITRE received responses from 141 
potential participants (5.64 percent) either via phone call or via the online form hosted at 
www.mitre.org. Additionally, MITRE created a waitlist for participant interviews for 
each site in case a minimum necessary number of interviews could not be conducted due 
to respondents’ failure to appear. Two participants failed to appear to their scheduled 
interview, and one participant cancelled approximately 24 hours in advance of their 
scheduled interview. MITRE called two candidates from the waitlist. Twenty-nine total 
interviews were deemed “completed,” meaning that the participants were eligible, and 
testing took place. 

3. Develop Free File program usability testing material. In preparation for the usability 
interviews, the MITRE team developed an interview protocol designed to have 
participants complete three tasks: 1) locating the Free File program online; 2) selecting a 
Free File software offer, including with the software lookup tool; and 3) completing a tax 
return using a Free File software offer chosen by MITRE. The MITRE team developed 
three fictitious scenarios for taxpayers to use while completing Tasks 2 and 3, in order to 
protect taxpayers’ privacy. Two scenarios were used to support tasks where taxpayers 
were asked to select a vendor for which a fictitious person was eligible (selection 
scenarios 1 and 2). The third scenario was used to support the task where taxpayers were 
asked to complete a tax return using a Free File software offer chosen by MITRE (filing 
scenario). To design the selection scenarios, MITRE examined eligibility criteria for 
different vendors and developed taxpayer characteristics that would return different sets 
of possible vendors. To design the filing scenario, the team had two goals: 1) to introduce 
enough complexity into the scenario so that taxpayers would have to interact with 
multiple components of the tax preparation software, including filing status, dependents, 
and adjustments to income, while still keeping the scenario simple enough to complete in 
approximately 30 minutes and 2) to ensure that scenarios represented feasible Free Filer 
characteristics. 

4. Conduct usability testing sessions. During the sessions, participants completed the 
fictitious scenarios to evaluate the experiences of completing three tasks: 1) locating the 
Free File program online; 2) selecting a Free File software offer, including with the 
software lookup tool; and 3) completing a tax return using a Free File software offer 
chosen by MITRE. MITRE researchers observed the session and then asked debriefing 
questions to establish what aspects of the Free File system (both the IRS.gov Free File 
pages and the specific vendor pages) caused confusion, errors, or frustration. In addition, 
MITRE also collected and analyzed data on taxpayer satisfaction and experience (via 
brief questionnaires), and taxpayers’ perceptions of the Free File system components and 
the Free File program. MITRE recorded the sessions with the participants’ permission 
(specifically, MITRE recorded audio and an image of the participants’ screens during the 
sessions). 
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5. Data analysis and reporting. MITRE summarized and analyzed the usability testing 
session data. MITRE applied aspects of behavioral science to interpret the session data in 
terms of assessing how taxpayers interacted with the Free File software programs. 

MITRE analyzed the interview data and developed key findings, summarizing the major 
themes and patterns of the interview data. The integrated findings from the taxpayer 
interviews are included in this report. 

2.7 Legal Analysis 

MITRE’s General Counsel conducted an analysis of the MOU to detennine where improvements 
could be made to improve both clarity and specificity, and provided a “red-lined” version of the 
MOU to illustrate suggested improvements. In addition: 

• An outside counsel retained by MITRE also simultaneously conducted the same analysis 
and red-lined the MOU. 

• MITRE’s counsel consulted with the IRS counsel to gain insight to the IRS perspective 
and discussed potential improvements to the MOU. 

• MITRE and the outside counsel collaborated to compare red-lines and come to consensus 
on the final red-line. 


PRE-DECISIONAL - FOR INTERNAL IRS W&l USE ONLY - NOT FOR PUBLIC RELEASE 

28 



PRE-DECISIONAL - FOR INTERNAL IRS W&l USE ONLY - NOT FOR PUBLIC RELEASE 


Findings 

3.0 Eligible Taxpayers and Actual Participation 


Because much of the criticism of the Free File program is based on what is perceived to be 
low participation, MITRE looked closely at the eligible taxpayer population to understand 
whether the criticism was warranted. While that determination could not be made without 
additional, unavailable data, what did become clear is that looking only at the number 
participating versus the number eligible is not a basis for the criticism. Highlights of this 
section include: 

• Although 70% of the population is technically eligible for the Free File program, 
that number is not representative of likely candidates. Half of those use a paid 
preparer, and more than 25 million taxpayers make other choices that take themselves 
out of the pool of likely candidates. 

• Many factors contribute to choices taxpayers make, and cost is only one. It cannot 
be assumed that simply because a program offered by the government is free, people 
will use it. Without a deeper analysis into taxpayer awareness and taxpayer behavior, 
the IRS should not assume any specific actions they take will increase participation. 

• According to the FFA, their members provided free commercial filings last tax 
season to nearly 18 million taxpayers. With free venues such as VITA, military 
assistance, TCE and free commercial filings, Free File is just one choice taxpayers 
have to file their returns for free. 


3.1 Overall Trends 

The Free File program met with considerable success in its first three years. During FS 2003— 
the first year of implementation—2.8 million returns were filed though 17 Alliance member 
offerings. While this represented less than 4 percent of eligible filers, the sheer number of Free 
Filings was considered a huge success given the novelty of the program. In its second year (FS 
2004), 3.5 million returns were filed (an increase of 26 percent), and in the third year (FS 2005), 

5.1 million returns (an increase of 46 percent) were filed. 49 50 FS 2005 marks the peak of 
participation. 

Data from IRS’ ETARAS shows an overall decline in the number of Free File accepted returns 
between 2008 and 2018, as shown in Figure 5. However, new data from the IRS CDW database 


49 M. S. Chu and M. M. Kovalick, "An Analysis of the Free File Program," in National Tax Associate Proceedings, 99th Annual 
Conference on Taxation, Boston, 2006. 

50 Source: IRS Electronic Tax Administration Research and Analysis System 
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shows that Free File numbers have started to pick up since TY 2016. Compared to TY 2017, 
approximately 74,000 more returns were filed to date for TY 2018. 

Since its inception, over 53 million returns were filed through the Free File program. The IRS 
reported that Free File saved taxpayers about $1.6 billion in tax preparation costs 51 . 
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Figure 5. Free File Accepted Returns, FY 2003-2018 

3.2 Taxpayer Behavior and Awareness of the Free File Program 

A key component of the Free File program is, of course, the taxpayer. In the early days of 
Free File, when encouraging electronic filing was one of the primary drivers, the IRS followed 
the patterns and behaviors of Free File participants closely through surveys, focus groups, and 
studies that looked at awareness and migration. 52 

As overall e-filing participation increased, and budgets for these types of studies decreased, the 
ability for the IRS to understand taxpayer behaviors declined. The last Free File awareness study 
was conducted in 2011. Because the environment has radically changed in the past eight years in 
terms of technology and filing choices available to taxpayers, the insights that study provided 
may no longer have value. 


51 Internal Revenue Service, "IRS, Free File Alliance announce changes to improve program; Improved taxpayer options 
available for 2019 Free File program," Internal Revenue Service, 2 November 2018. [Online]. Available: 
https://www.irs.gov/newsroom/irs-free-file-alliance-announce-changes-to-improve-program-improved-taxpayer-options- 
available-for-2019-free-file-program. [Accessed 2 July 2019]. 

52 Internal Revenue Service, "Free File Survey, Publication 4556 (11-2008), Catalog Number 48884," Internal Revenue Service, 
Washington, D.C., 2008; Internal Revenue Service, "Findings From The Free File Migration Study, Publication 4684 (3-2008), 
Catalog Number 51580P," Internal Revenue Service, Washington, D.C., 2008 Internal Revenue Service, "Report of Findings 
From the Free File Marketing Focus Group, Publication 4833 (Rev 09-2010) Catalog Number 54427P," Internal Revenue 
Service, Washington, D.C., 2010;Intemal Revenue Service, "Free File Demographics and Migration Analysis, Publication 4829 
(Rev 11-2011), Catalog Number 54376," Internal Revenue Service, Washington, D.C., 2011; Internal Revenue Service, "Free 
File Awareness & Attitudinal Research, Findings From Research Conducted Among Free File Non-Users With Focus On Three 
Specific Segments, Publication 4928 (01-2011), Catalog Number 57580X," Internal Revenue Service, Washington, D.C., 2011. 
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Based on the participation rates, it would appear that awareness of the program is low, though 
the 2011 study and other e-file studies indicate awareness is not the only factor that determines 
usage of the program. While the IRS has continued to collect data on demographics and 
migration of Free File users, no recent— if any— comprehensive studies on taxpayer behavior 
(i.e., why taxpayers make the choices they do) have been conducted. Understanding the numbers 
is instructive and Section 3.3 examines those numbers more closely. However, until the IRS 
understands why taxpayers make the choices they do, they can only make programmatic 
decisions based on assumptions. A behavioral study helping the IRS understand the choices 
taxpayers make in each of the demographics, along with an updated awareness study, could help 
the IRS determine how to improve participation, if that is the goal. 

3.3 A Closer Look at Eligibility and Participation 

As mentioned earlier, we must look at Free File in the context of taxpayer behavior and taxpayer 
choices. One cannot assume that everyone in the eligible population would use Free File if they 
were aware of it, nor can we assume how many people should use it. But we can narrow down 
the population of taxpayers who may be more likely to use it if they know about it. 

Taxpayers have several methods to choose from when filing taxes— the IRS allows tax return 
submission via paper and electronic forms. Taxpayers’ decision to choose between self¬ 
preparation and a paid tax professional factors in the financial costs as well as the time, effort, 
and hassle associated with each filing method. Figure 6 shows a simplified taxpayer decision tree 
for choosing different methods of filing and summarizes the associated economic costs. 

The economic factors of making those choices are discussed in greater length in Appendix A. 
Understanding these factors are important when attempting to determine who should use 
Free File, who does use Free File, and how the IRS might increase the awareness of Free File 
(and therefore, presumably, the participation). 

As stated earlier, the IRS detennines who is eligible for the Free File program by looking at prior 
year returns and taking the lower 70 percent by AGI. For TY 2017, nearly 104 million taxpayers 
were eligible for the program by having an AGI of less than $66,000. Less than 3 million 
actually used the program, which has brought the program under the recent scrutiny described in 
Section 1.1.4. 

But being eligible and being an actual candidate for the Free File program can be very different. 
As demonstrated in Figure 6, taxpayers have many choices when they file their returns, and 
many of those choices will take them out of the likely candidate pool for Free File. 

For example, taxpayers generally have a preference to whether they e-file or file on paper, and 
whether they prepare the return themselves or go to a paid preparer. Because Free File is a DIY 
tax filing program, a valid comparison can only be made to the overall e-filing DIY population, 
i.e., do “self-prepared” taxpayers pay for the software and e-filing, or do they look for a free 
version? 
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Economic Costs by Filing Method 




Figure 6. Economic Cost by Filing Method. 

There are other free venues that a taxpayer could select, such as using a free commercial product, 
getting free assistance through a Volunteer Income Tax Assistance (VITA) or Tax Counseling 
for the Elderly (TCE), or through the many free options available for active military at military 
bases. Additionally, many taxpayers choose to receive their refund immediately either through a 
Refund Anticipation Check (RAC) or a Refund Anticipation Loan (RAL), which has not been 
allowed through the Free File program since the 2007 agreement. In TY 2017, of the eligible 
population, more than 18 million taxpayers received a RAC or RAL, nearly 10 million of those 
were in the self-prepared group. And, of the overall population of eligible taxpayers, nearly 12 
million chose to file on paper that year, with 8.6 million of those taxpayers being in the group 
who self-prepared. While some paper filers, particularly younger and first-time filers are 
traditional target candidates to become e-filers/Free Filers, if we look at TY 2017 retrospectively 
(as shown in Figure 7), making a choice to file on paper removed those filers from the candidate 
pool for Free File that year (i.e., keeping the comparison to DIY e-filers). 

Looking more closely at the filing characteristics of this population demonstrates that while the 
IRS clearly has opportunity for increasing the participation in the Free File program, that 
opportunity may come from a much smaller population than the actual number of eligible 
taxpayers. As such, Figure 7 draws a more accurate picture of that population for TY 2017. 
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103,778,000 Eligibile Taxpayers - TY 2017 



■ Paid Preparers 

■ RAC/RAL self-prepared 

■ Paper filers - self-prepared 

■ VITA/TCE 

■ Free Filers 

■ Potential Add'l Free Filers 


Figure 7. Break-down of 103,778,000 Taxpayer Eligible for Free File (TY 2017). 

However, the pool of potential Free Filers shown in Figure 7 is actually smaller than the almost 
28,000,000 noted, as most commercial software companies offer a free version of their product 
for the simplest returns outside of the Free File program. That information is not publicly 
available, though MITRE requested the number of returns filed through a free commercial 
offering by the members as an aggregate from the FFA, who attested to 17.7 million for TY 2018 
from 11 current members. 

An u nkn own number of taxpayers presumably used free commercial offerings from software 
companies outside of the Free File Alliance, which would make the number of users of free 
commercial offerings even larger. 

While MITRE cannot definitively say whether or how many taxpayers in the groups outside of 
the “additional potential free filers” in Figure 7 could or should have been Free Filers, what the 
chart reiterates is that taxpayers have many choices when they file their returns. And some of 
these choices are free, outside of the Free File program. If the focus is on ensuring taxpayers who 
are least able to pay for tax preparation and filing are getting free help, then it raises the question, 
should the objective be to increase the participation in Free File program, or more broadly in free 
filing. 

3.4 Free File and VITA 

In breaking the eligible tax filing population of the 103.8 million into those who use a return 
preparer and those who self-prepare, a useful comparison may be made between those who pay 
for a preparer and those who choose VITA, an assisted free option, to the paid DIY and those 
who choose Free File. The VITA program has 2,400 partner organizations spanning government, 
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corporate, educational, and non-profit, including very large organization such as AARP who 
advertise their more than 10,000 tax-help sites nationwide. The IRS has offices and employees 
nationwide to administer the VITA program, and thousands of professional volunteers who work 
at the sites each year, yet the number of those choosing this free program is roughly equivalent to 
those choosing the Free File program (see Figure 8). 



Volunteer Income Tax Assistance 

(Bricks and Mortar Assisted Free 

Tax Filing Program) 

Free File Program 

(DIY Free Tax Filing Program) 

Number of taxpayers within 
the targeted (103.78 million) 
population (Uses paid 
preparer or self-prepares) 
eligible for assisted help 

Access to Program 

Program Administration 

Number of Users TY2017 

51.86 million using paid preparers 
Less 3.4m paper filers 

Less 8.4m RAL/RACs = 

40.06 million returns* 

11,000 locations nationwide 

offices and employees nationwide, 
>2,000 partner organizations 

~ 3 million 

48.82 million Self prepared 

Less 8.75m paper filers 

Less 9.75m RAL/RACs = 

30.32 million returns* 

Choice of 12 companies through 
IRS Free File Landing Page 
program office in HQ, Alliance 
of 12 software companies with 
Executive Director 

~ 2.6 million 

Ratio of free to paid 

7.5% 

8.5% 


* Because the comparison is being made between e-filed returns, paper returns are backed out of the population, as are RACs and 
RALS, both of which are unavailable for VITA and Free File 

Figure 8. Comparison of Two Free Programs: VITA (Preparer-assisted) and Free File (DIY). 

As noted earlier, without further research to understand taxpayer behavior and what drives 
taxpayers to make the choices they do, it’s difficult to know whether the percentages of those 
seeking free tax return help or free filing is reasonable or if it’s too low. As there are many 
unknowns about taxpayer behavior and choice that will require further research, and these 
questions align with MITRE’s internal research program initiatives on improving customer 
service, MITRE will consider this for future research. 
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4.0 Compliance with the MOU 


The purpose of this chapter is to provide a comprehensive assessment of current state 
compliance with the Free File agreement; that is, individual Free File member organization 
and IRS Free File program compliance with the most current MOU agreement (October 31, 
2018). The assessment includes findings from the IRS interviews and document reviews, 
interviews with Alliance leadership and Alliance-contracted reviewer, and fact-finding 
interviews with representatives from each of the Free File member organizations. In addition, 
MITRE conducted hands-on reviews of member Free File websites (taxpayer journey) and 
web forensic analyses. See Section 2.0 for methods used for each of these activities. 

Key themes from this set of findings include: 

• MOU is a living document that has evolved over time. The original agreement 
between the IRS and the software industry has been updated numerous times since the 
program’s inception. While key aspects in the MOU have remained intact, as issues 
come up, IRS and FFA negotiate resolution and amend the agreement accordingly. 

• Overall, FFA members are generally compliant with the MOU as written. 

Assessing against current MOU provisions—and at the time of the assessment— 
MITRE found only a few requirements inconclusive, non-compliant, or not applicable. 

• Search marketing is an emerging issue that is not explicitly addressed by the 
current MOU. One of the issues at the core of recent criticisms is whether members 
employed—or should be able to employ—search routing techniques. For the next 
iteration of the MOU, IRS has the opportunity to provide clear guidance in this area. 

• The IRS and FFA work together to ensure MOU compliance. While there is room 
for improvement, based on assessment findings, MITRE considers the IRS’s current 
compliance processes adequate and effective to support the integrity of the program. 

In part, this is the result of its long-standing partnership with FFA and continuous 
assessment and improvement of the program as it has evolved. 


4.1 MOU Provisions Set Criteria for Compliance 

As a result of the most recent MOU negotiation in October 2018, new and revised provisions 
were added for implementation in FS 2019. According to the IRS press release announcing the 
updates, these changes to the agreement were aimed at making the Free File program “more 
taxpayer friendly while strengthening consumer protections in several key areas,” 53 presumably 
to address public criticism outlined in Section 1.1.4. Based on this updated MOU—which will be 


53 Internal Revenue Service, "IRS, Free File Alliance announce changes to improve program; Improved taxpayer options 
available for 2019 Free File program," Internal Revenue Service, 2 November 2018. [Online]. Available: 
https://www.irs.gov/newsroom/irs-ffee-file-alliance-announce-changes-to-improve-program-improved-taxpayer-options- 
available-for-2019-free-file-program. [Accessed 2 July 2019], 
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in place until the next round of negotiations slated for October 31, 2021—Free File member 
organizations must adhere to the following additional requirements (beginning in FS 2019): 

• Remove any “value-added” button from Free File member landing pages. Free File 
member landing pages cannot include a value-added button—defined as “an icon, link or 
any functionality” that direct taxpayers to non-Free File programs (commercial products 
or paid services). [MOU Section 4.32.6 54 ]. 

• Provide an option for taxpayers to “easily return” to the IRS Free File page if they 
don’t qualify for a particular member offer. If taxpayers do not qualify for members’ 
Free File offer (i.e., due to the different eligibility characteristics beyond the $66,000 AGI 
limit), members must allow taxpayers the option to first return to the IRS landing page on 
IRS.gov to see if they qualify for another member’s Free File offer. After that, the member 
can offer two additional options, in this order: 

o If taxpayers are eligible for Free File (but not the particular member offer), a second 
option is offering a free alternative to complete their return with the same member, 
o If taxpayers are not eligible for Free File (e.g., has AGI over $66,000), a third option is 
to continue to the member’s commercial site to complete their return for a fully 
disclosed/ee. (MOU Section 4.19.2) 

These options are shown graphically Figure 1 in Section 1.1.1. 

• Offer returning Free File taxpayers the option to use Free File again when they log 
into member’s website. If taxpayers return to a Free File member’s commercial website 
after using the Free File program the previous year, the member must first offer taxpayers 
the Free File option for the current filing season when taxpayers log in to their account. 
The option to continue with the Free File program must be offered before any alternative 
commercial products or services. ( MOU Section 4.14) 

• Send at least one follow-up email to taxpayers who used Free File the previous year 
to “welcome them back to Free File service.” For taxpayers who participated in the 
Free File program the previous year, members must send at least one email to remind the 
taxpayer about the availability of the Free File program and invite them to return to the 
member’s Free File landing page. The email communication cannot include offers or links 
to non-Free File programs (commercial products or paid services). (MOU Section 4.32.4) 

• Participate in-place review process. The latest MOU includes updates to the already 
existing annual review process conducted by the IRS and the Free File Alliance. The 
MOU explicitly calls out pre-filing season review of members’ landing pages and an 
additional review conducted by the IRS that is a random, unannounced review during the 
filing season to ensure continued compliance. ( MOU Section 4.29.1 and Section 6.1.3) 

Evolution of Agreements and MOU 

The changes to the MOU implemented in FS 2019 are aligned with what MITRE observed when 
reviewing the historical context of the MOU (and overall agreements between the IRS and 


54 Section 4.32.5 further clarifies “No Other Sales and Selling Activity” - limiting “marketing, soliciting, sales or selling activity, 
or electronic links to such activity” to paid federal return as described in the third option for ineligible taxpayers in the next bullet 
[Section 4.19.2] and state tax preparation offers as described elsewhere in the MOU. 
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Free File Alliance). Since the Free File program began in 2002, there have been six fonnal 
agreements between the two parties, corresponding to eight iterations of the MOU, as shown in 
Table 4. Each time there is a new agreement or updated MOU there are associated changes to 
terms of service—either to address an issue current at the time, or general update to reflect the 
current enviroment. 

It is important to note that with each iteration, additional infonnation is added to—or in some 
cases removed from—the MOU. The numbering and potentially sequence of the provision 
changes, but the foundation of the MOU stays the same. In this way, the MOU is iterative; there 
is no wholesale “redo” of the MOU. 

For example, from the original agreement in 2002 to the first MOU in 2005, limits were placed 
on members’ offering RALs and RACs. In this first mention of RALs and RACs, it is agreed that 
they are allowed, but with restrictions. However, starting in the second MOU in 2007, RALs and 
RACs are prohibited within the Free File program. To address this change, Section 4.2 of the 
MOU is shown as “Ommited by Intention.” That language is carried through to the most current 
MOU. 

Conversely, there are sections of the MOU that have not changed from year to year. For 
example, after a slight revision to how the program objectives are characterised from the first 
agreement to the first MOU, the MOU continues to contain the same program objectives. See 
Section 4.3 for additional discussion of the Free File program objectives. 

Similarly, there are at least two foundational statements in the MOU preamble that were in the 
first version of the MOU and is still in the most recent version: 

• Preamble. “[I]t was agreed that the Alliance will offer the Services and the IRS will 
provide taxpayers with li nk s to the Services offered by the Alliance participants offered 
through a web page, which is hosted at irs.gov with links from www.usa.gov.” 

• Article 2 FFI and IRS objectives. “In recognition of this commitment, the federal 
government has pledged not to enter the tax preparation software and e-filing services 
marketplace.” 


Table 4. Free File Program Agreements and MOUs. 


Year Free File Agreements and MOUs 


Examples of Updates 


2002 


2005 


• First Agreement between the IRS and 
Free File Alliance, LLC executed on October 
30, 2002, to launch and set parameters for 
the IRS Free File program (three-year 
agreement) 

• Second Agreement between the IRS and FFA 
executed on October 31, 2005, to extend and 
amend original (four-year renewal 
agreement) 

• IRS and FFA enter into first MOU as of 
December 20, 2005 to govern partnership for 
implementation in FS 2006 


• Guarantee coverage in aggregate to 60 
percent taxpayers 

• Each member provides minimum of 10 
percent coverage 

• MOU to administer program 

• Guarantee coverage in aggregate to 70 
percent taxpayers (up from 60 percent) 

• No member can cover more than 50 percent 

• Begin using AGI 

• Members will provide Free File indicator 

• Limits (but does not prohibit) RALs 
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Year 

2007 

2008 

Early 2009 


Late 2009 


2014 


2015 


2018 


Free File Agreements and MOUs 

• IRS and FF enter into second MOU as of 
January 12, 2007 for implementation in FS 

2007 with revised and additional 
requirements for Free File program 

• IRS and FFA enter into third MOU as of 
January 2, 2008, for implementation in FS 

2008 with revised and additional 
requirements for Free File program 

• IRS and FFA enter into fourth MOU as of 
January 13, 2009 for implementation in FS 

2009 with revised and additional 
requirements for Free File program 

• Third Agreement between the IRS and FFA 
executed on October 30, 2009, to extend and 
amend the previous (five-year agreement) 

• IRS and FFA enter into fifth MOU as of 
October 20, 2009 for implementation in FS 

2010 with revised and additional 
requirements for Free File program 

• Fourth Agreement between the IRS and FFI 
effective October 30, 2014, with the intention 
to enter into a multi-year agreement (one- 
year agreement) 

• IRS and FFI enter into sixth MOU signed in 
conjunction with 2014-2015 agreement until 
next multi-year agreement signed 

• Fifth Agreement between the IRS and FFI 
effective October 31, 2015, to extend 
previous (five-year agreement) 

• IRS and FFI enter into seventh MOU signed in 
conjunction with 2015 agreement with 
revised and additional requirements for 
Free File program 

• Sixth—and most current—Agreement 

between the IRS and FFI effective October 31, 
2018, to replace 2015 agreement (five-year 
agreement) 

• IRS and FFI enter into eighth—and most 
current—MOU signed in conjunction with 

2018 agreement for implementation in FS 

2019 

• Both the 2018 agreement and eighth MOU 
are set to expire on October 31, 2021. 


Examples of Updates 

• Prohibits RALs/RACs, provisions removed 
from MOU 

• Identifies permitted sales, limits on ancillary 
sales, and selling activity 

• Adds implementation of CAPTCHA 

• Adds third-party certification of returns as 
part of already existing guarantee of 
calculations 

• Adds unbranded tillable form utility to MOU 
(specifications dated December 9, 2008) 

• Adds basic package of forms and schedules 
required for all members 

• Adds core forms and schedules 

• Additional guidance under development for 
pop-ups, spyware, and marketing tools 

• Adds customer satisfaction survey 

• Additional security and privacy requirements, 
including compliance with e-file standards 

• Linkonlandingpagebacktoirs.gov 

• Free File Alliance officially changes name of 
organization to Free File Inc. 

• Adds annual review and revision of MOU 


• Members to offer the same federal forms and 
schedules as basic commercial online product 

• Promotion of Free File program responsibility 
of IRS 

• Adds language around innovations, included 
pre-population of tax information and forms 

• Removal of value-added buttons from 
member landing pages 

• Define paths for taxpayers who are not 
eligible for a member offer (i.e., first back to 
IRS.gov, then free alternative, then 
commercial) 

• Adds requirements for engaging returning 
Free File taxpayers (i.e., offer Free File at log¬ 
in, email taxpayers) 

• Refines requirements for existing IRS and FFI 
review processes 
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4.2 Free File Alliance Member Adherence to MOU 

This section summarizes MITRE’s assessment of each Free File member organization’s 
adherence to the current set of MOU requirements, including the latest additions implemented in 
FS 2019 (outlined in the previous section, Section 4.1) 

Section 4.3 describes the IRS Program Office role in assuring adherence to the MOU, including 
implementation of the updated review processes also included in the FS 2019 MOU 
requirements (Section 4.1). 

4.2.1 FFA Member Assessment Results: Overview 

Overall, members were generally compliant with the MOU, in particular with Article 4, 
Standards of Practice, which outlines the expected level of service, including software and 
website functionality, security, disclosure of forms and schedules as well as taxpayer service 
options and other requirements such as innovation. More specifically, MITRE found in assessing 
72 MOU requirements for each FFA member only a few were either deemed as inconclusive, 
non-compliant, or not applicable. 

As mentioned in Section 2.0, MITRE not only collected data from each of the members, either 
through in-person or telephone interviews or through written responses to a specific set of 
questions, but also through stakeholder interviews and other analytics methods involving web 
reviews and taxpayer journey evaluations. FFA member compliance to the MOU was assessed 
using several methods. Methods used included: 

• Interviews with member companies (site visits). Responses given during the interviews 
or provided in writing. These inputs are considered self-attestation. 

• Taxpayer journey reviews for each member site. Results from landing page and 
comprehensive site reviews. These inputs are considered observable as they can be seen 
while interacting with member sites. 

• Other stakeholder interviews. Responses given by FFA leadership or key IRS 
stakeholders. 

MITRE also looked at the results of the IRS website reviews (see Section 4.3 for more 
information) as a supplemental input to determine if more infonnation was needed. 

Specific Instances of Non-Compliance 

There were two instances where members were determined to be non-compliant with certain 
provisions of the MOU. They include: 

• One member was non-compliant with a sentence in MOU 4.15.4. The MOU states that 
“Members must clearly list their free customer options ... on the Member’s Free File 
Landing Page (or such page must have a clear and prominent link to such disclosures 
directly from this page).” Based on MITRE’s landing page analysis, the link was not 
“clear and prominent” on their landing page 55 . 


55 As of the date of this report, the member has updated their Free File landing page. 
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• One member was non-compliant with MOU 4.32.4. The MOU requires members to 
“communicate not less than once annually via email with their taxpayer customers who 
used Free File services and completed their returns through Free File in the immediately 
preceding tax year.” The representative from the member acknowledged they were 
supposed to initiate an email to their prior tax season Free File taxpayers and self-attested 
that they did not provide the required email prior to the tax season. 

Instances Deemed Inconclusive or Not Applicable 

The following MOU requirements from Article 4 were deemed as inconclusive or not applicable 
for specific members, they include: 

• 4.15.4 Disclosure of Taxpayer Service Options (inconclusive) - (applicable to 3 
members). The MOU states that “Members must provide taxpayers a free electronic 
method to obtain a copy and learn the status of their electronically filed return.” Based on 
what could be observed on three member sites through taxpayer journey activities—which 
were limited to all steps prior to actually submitting the test return—MITRE was unable to 
verify that these members offer a method to check the status of the return without first 
submitting a return. 

• 4.19.2(i -vi) Eligible taxpayers/no promotional codes or rebates/links to paid sites 
(inconclusive) - (applicable to 2 members). For this set of provisions, the MOU states 
that member programs must “unequivocally inform taxpayers who are ineligible for the 
free offer at the earliest feasible point,” and outlines a set of requirements specific to when 
a taxpayer is informed of ineligibility (e.g., reason for ineligibility, alternative actions that 
must be presented in a specific order, etc.). During the taxpayer journey analysis, MITRE 
tested a scenario in which a taxpayer has an AGI over the eligibility limits. For two 
members, testers in this ineligible scenario were able to prepare their Free File tax return 
without receiving ineligibility notification. In both cases, the testers reached the signature 
page where the next step was to file the return and were unable to verify whether or not 
the appropriate notifications, reasons, alternative actions, etc. were provided. The two 
member programs varied slightly in that: 

o One member has a wizard on its landing page that allows a taxpayer to determine 
eligibility prior to creating an account. This optional step guides a taxpayer through 
eligibility criteria and does provides appropriate language if they are ineligible. 
However, this is not using actual taxpayer information and if the taxpayer ignores the 
landing page wizard—or does not see it— or guesses information incorrectly, the 
software allows the ineligible tester to continue to the point of the signature page, as 
described above. 

o The other member does not have a wizard but does have generic language on the 
landing page describing their offering and eligibility requirements. Presumably, a 
taxpayer could determine eligibility from this though it is unclear whether this 
constitutes ineligibility notification “at the earliest feasible point.” Again, this 
member’s software also permitted the ineligible tester to continue to the signature page 
with no notification. 

MITRE did receive screenshots from the IRS that were provided by all members on this 
issue. Based on the screen shots, it appears taxpayers would receive an ineligibility 
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message and the members do have the appropriate language, but it is unclear when these 
messages are provided, and MITRE could not independently verify this. As explained 
above, since the MITRE assessment was limited to steps prior to actually submitting the 
test return, MITRE was unable to observe any messages that may have been presented 
after submitting the return. MITRE testers were therefore not able to determine whether 
reasons and alternative actions were proper. These two members were therefore 
inconclusive in all “ineligible” categories. 

• 4.32.2 Permitted sales, limits on ancillary sales, and selling activity (inconclusive)- 
(applicable to 2 members). The MOU states “when a taxpayer enters a Member’s Free 
File Landing Page and begins to complete a return but ultimately cannot qualify for the 
Member’s free offer, the member must provide, as a first option, a prominent hyperlink for 
the taxpayer to return to the IRS Free File Landing Page (consistent with Paragraph 
4.19.2)” The remainder of the paragraph outlines the order of other options and stipulates 
that the “charge for such commercial service shall not exceed the usual commercial price 
for such products or services.” For the same rationale provided for Paragraph 4.19.2, 
compliance for two members was deemed inconclusive, subject to discussion with IRS. 

• 4.34 Use of Free File Logo (not applicable) - (applicable to 2 members). This is a 
requirement that is only applicable if there is a logo. Since these members did not use a 
logo, it is not applicable. 

• 4.36.3 Innovations (not applicable) - (applicable to 1 member). Because this is 
between the IRS and FFA, it is not possible to rate compliance of individual members. 
Nevertheless, all but one member detailed during interviews how they have introduced 
multiple and continuing innovations into both products; some noted they were including 
capabilities offered to all FFA members by one member. This member self-attested that it 
didn’t have the resources to pursue major innovations. 

MITRE was unable to verify compliance for the following provisions of the MOU and 
detennined inconclusive for all members. They were: 

• 4.17.3 Blackouts (inconclusive). Based on stakeholder interviews with the IRS and FFA 
along with members’ input, there were no unplanned blackouts. As a result, maintaining a 
consistent level of service could not be determined nor whether customers seeking access 
to the Free File option were directed elsewhere, in particular to the fee-based services of 
the member. Hence, MITRE rated the evidence for all members as inconclusive. 

• 4.19.2(iv) Eligible Taxpayers/No Promotional Codes or Rebates/Links to Paid Sites 
(inconclusive). MITRE did not explicitly verify whether the messaging, language, and 
formatting relating to disqualification practice by members adhered to a specific standard 
provided by the FFA ED in consultation with the IRS. This was not covered during site 
interviews. MITRE did observe that all members—with the exception of the two 
previously discussed members—had messaging with regards to disqualification, but 
MITRE did not compare against a standard. If no standard language exists, the MOU 
should be updated to reflect the guidance that is provided. 
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4.2.2 Emerging Issues: Search Techniques 

As discussed in Section 4.1, the MOU is an evolving agreement, changes to which are made 
through negotiations between the IRS and the Free File Alliance at specific points in time and/or 
as new issues emerge. 

The assertions from ProPublica and to some extent recommendations from IRSAC and TAS 
opened the door to discussion around the broad topic of pay-per-click search advertising and 
search engine optimization techniques (collectively known as search marketing) that enable web 
publishers to acquire large amounts of traffic from the major web search engines and exert 
control to which pages on their websites this traffic is directed to. 

There is no place in the MOU that explicitly addresses the issues related to search marketing or 
provides guidance to the members for standards of practice. We have provided the discussion 
here—including our data and analysis—for the IRS to consider for the subsequent MOU 
negotiation, currently planned to result in an updated MOU by October 2021. 

4.2.2.1 How Taxpayers Find the Free File Program 

As shown in Table 5, there are three primary ways for users to get to the IRS.gov Free File 
landing page: web search engines such as Google and Bing, clicks from referral li nk s on other 
third-party websites, and direct traffic (or traffic from unknown sources). 

Table 5. Top Sources of Traffic to the Free File Member Landing Page. 


Top Sources of Traffic for the 

FFI Landing Page 

Entrances 

Percent of Total 

Organic Search 

1,541,816 

55.2% 

Referrals from third-party websites 

761,258 

27.3% 

Direct or unknown sources 

476,060 

17.1% 


Web Search Engine Activity 

MITRE conducted website traffic analysis using data downloaded from SEMrush 56 , a digital 
market software as a service company. This analysis examines searcher behaviors and the 
numbers of taxpayers that were searching for free tax filing services from a major web search 
engine such as Google. It also provides analysis of the seasonal nature of the search activity 
relating to free tax filing services. 

To help understand the search discussion, it may be helpful to define a few terms. (Also see 
Appendix E for an overview of how search engines work). 

• Organic search. The primary set of search results on major web search engines such as 
Google and Bing that use an algorithmic approach to ranking and displaying search results 
to users. This result type is explicitly not influenced by payments from web publishers. 


56 SEMrush Inc., "All-in-one Marketing Toolkit," SEMrush Inc., 2019. [Online], Available: https://www.semrush.com/. 
[Accessed week of 27 May 2019], 
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• Paid search. Unlike organic search results, paid search results are explicitly 
advertisements that web publishers purchased on an auction-like bidding platform in 
which search ads that have high bid prices and have been shown to be relevant to the 
user’s search query will obtain the most prominent positions on the search results page for 
a specified search query. 

• Keywords. This is shorthand for “search query.” It is the word or phrase a user types into 
the search box to express what they are looking for. “Search term” is another synonym of 
keyword and search query. 

What search terms did taxpayers use to find Free Tax filing services? 

MITRE identified a representative sample 108 search queries (keywords) showing the range of 
pennutations used by taxpayers to search generically for free tax filing services. As discussed in 
Section 3.2, a few criteria were used to refine this list of keywords to only those keywords 
considered to be highly relevant to this service. There are likely many more less relevant 
keywords that also drove traffic to these pages and are not accounted for by this list of 108. 

As shown in Figure 9, of the 108 keywords, the top five accounted for 82.5 percent of all organic 
search traffic to the FFA member sites and IRS.gov. The top keyword—“free tax filing”—drove 
63 percent of all organic search traffic. 


Top Keywords for FFA Landing Pages 

Est Total Traffic 
During FS 

Percent of 
Total 

free tax filing 

1,177,297 

63.1% 

free online tax filing 

146,135 

7.8% 

free taxes online 

106,560 

5.1% 

free tax return 

65,437 

3.5% 

free file 

56,214 

3.0% 


Figure 9. Top Keywords for Member Free File Landing Pages 57 . 

Following a free tax filing search, how many taxpayers clicked on paid vs organic search 
results? 

In total, using the list of 108 free tax filing keywords, MITRE estimated over 12 million visits 
went from Google to the FFA member sites and IRS.gov. Of those visits, it is estimated that 85 
percent of the traffic followed a paid search result while the other 15 percent of visits came from 
an organic search result. Searchers were more than five times likely to click on a paid search 
result. 

According to the third-party data source used for this analysis, the average cost per click for a 
free tax filing search advertisement was $1.69. This results in an estimate of $17.3 million for 
total advertising spent during the 2019 tax filing period. 


57 SEMrush Inc., "All-in-one Marketing Toolkit," SEMrush Inc., 2019. [Online]. Available: https://www.semmsh.com/. 
[Accessed week of 27 May 2019], 
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How much of the search traffic went to one of the FFI landing pages vs other “commercial 
type” products that offered “free tax filing” services? 

Of the 1.9 million visits that went from Google organic search to an FFA member site, 56,000 (3 
percent) went to an FFA landing page operating within the IRS Free File program. This is 0.5 
percent of all search visits (including paid) to the FFI websites for the 108 free tax filing 
keywords. 

Table 6 shows the estimated traffic distribution from organic and paid search results to each of 
the 12 FFA member sites. The first two data columns are focused on estimated traffic coming 
from organic search results. The third data column is an estimate of visits from paid search 
results. Notice that there is no column for estimated paid traffic to FFA landing pages. This is 
because the data indicated that none of the FFA members purchased any paid search 
advertisements to take taxpayers to their FFA landing page. However, during the FFA member 
interviews, one member stated that their company does employ advertising practices for the Free 
File program 58 . The fourth data column estimates the total money spent by the FFA member 
using an estimate of $ 1.69 per click. 

Table 6. Estimated Traffic Distributions from Organic and Paid Search Result to Landing Pages/Estimated 

Cost for Paid Traffic 59 . 


FFI Member Sites 

(Member names redacted) 

Est Traffic to 

FFI Landing Pages 

Est Traffic to 

Non-FFI Landing Pages 

Est Paid Traffic to 
Non-FFI Landing Pages 

Est Cost 
for Paid Traffic 


0 

3,823 

246,856 

$416,912 


378 

248 

0 

0 


648 

65 

0 

0 


0 

11 

54,594 

$91,616 


0 

0 

0 

0 


3,370 

0 

0 

0 


0 

697,756 

5,400 

$103,270 


3,488 

95,688 

1,812,769 

$3,060,979 


205 

179,161 

3,662,118 

$6,213,910 


11,696 

562 

0 

0 


36,310 

9,547 

3,039,163 

$5,125,896 


0 

40,759 

1,446,217 

$2,426,933 

Total 

56,095 

1,812,218 

10,267,117 

$17,347,900 

IRS.gov (for comparison) 

700,780 

83,819 

0 

0 


58 During MITRE interviews with representatives of FFA member organizations, one member indicated that their company pays 
for advertising of the Free File program through a number of methods. However, the interview did not include detailed discussion 
of advertising practices specific to key word search. For this analysis, MITRE looked specifically at ads purchased for key words 
related to free tax filing. The results of this analysis indicated that no purchased ads directed users to the landing pages of Free 
File offers. 

59 SEMrush Inc., "All-in-one Marketing Toolkit," SEMrush Inc., 2019. [Online], Available: https://www.semrush.com/. 

[Accessed week of 27 May 2019], 
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What is the seasonal nature of searches for “free tax filing?” 

Using the search term “free tax filing” as highly representative of the entire collection of 108 
keywords, MITRE submitted this as input to Google Trends to analyze the seasonality of 
searches for this service. Figure 10 shows that there is a substantial peak in search activity for 
these terms stretching from the end of January to the middle of April. This is completely 
consistent with what is referred to as tax filing season. Additionally, MITRE estimates that tax 
filing season search activity for people searching for free tax filing services represents 90 percent 
of total search traffic for the entire year. 



• free tax filing 


United States, Past 12 months 



Figure 10. Seasonality of Free File-Related Searches 60 . 


4.2.2.2 Search and MOU Compliance 

MITRE’s source code analysis of member Free File landing pages found that five of the 12 
companies engaged in a search routing practice that prevented their Free File offering from 
appearing in the organic search results of the major web search engines—the use of rneta robots 
NOINDEX (M)—or the rel=canonical tagFree File (R). Two companies changed the coding after 
the ProPublica articles. (See Appendix D for complete source code explanations and findings on 
search engine avoidance and Appendix E for how major search engines work.) 


60 Google Trends, 2019. [Online], Available: https://trends.google.com. [Accessed week of 27 May 2019], 
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When questioned about the practice, most members reported they believed that excluding their 
Free File landing pages from search results kept them in accordance with the MOU. The two 
companies that removed the coding said they still believe excluding the pages was appropriate, 
but after it was raised publicly, didn’t want to create the perception that they were doing 
something wrong. 

The MOU states in the preamble that... “the Alliance will offer the Services and the IRS will 
provide taxpayers with links to the Services offered by the Alliance participants...” While the 
MOU doesn’t expressly prohibit taxpayers from accessing Free File directly from the members’ 
Free File landing page, the IRS stated in a news release announcing the opening of the Free File 
program: “The free online software program, accessible only through IRS.gov, ...” This is what 
members said led them to exclude their own landing pages in an organic search, so that taxpayers 
wouldn’t come directly to their company Free File landing pages. 

To illustrate this point, Figure 1 from Section 1.1.1 was annotated to show the MOU-specificd 
path to Free File offerings. If Free File is only accessible through IRS.gov, taxpayers looking for 
the program would have to either go directly from IRS.gov or go to IRS.gov from a web search 
like Google or Bing. Taxpayers would not be able to go from other websites (like members’ 
commercial sites) nor would they be able to use a web search to bypass the IRS.gov landing page 
to go directly to a member’s free file landing page. 


Taxpayer selects 
Free File program 
landing page 


Taxpayer selects a 
Free File offer based 
on eligibility criteria 


Taxpayer files taxes for 
free using IRS Free File 



Figure 11. The “MOU-specified” Path of Free File Through IRS.gov 61 . 

ProPublica asserted the exclusion was used to drive searches for free tax filing to the companies’ 
commercial offerings, which could result in taxpayers paying for services, though they were 
eligible for free tax filing. MITRE’s web analysis of organic searches shows the vast majority of 


61 Icons created by Vectors Point, Lil Squid and monkik from Noun Project. 
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those searching for free tax filing landed on commercial sites, and no searches landed on sites of 
the five companies who excluded their Free File landing pages, though more searchers landed on 
IRS.gov’s Free File landing page than any individual members’ commercial page (for unpaid 
traffic only—four members’ web traffic exceeded IRS.gov traffic with paid searches). 

It must also be pointed out that excluding the Free File landing pages may not have translated 
into fewer Free File returns for those companies. Of the five companies who used the 
NOINDEX, four increased their free filed returns in the 2019 filing season, and one remained 
relatively flat. Of the seven companies who did not exclude their pages, five had increases in 
Free File returns and two had decreases. While it’s impossible to know whether the increases 
would have been greater had the landing pages not been excluded, three of the five companies 
who excluded had larger increases in Free File over the previous year. 

MITRE is not taking a position on the use of NOINDEX. If the IRS wants taxpayers to find 
Free File only through IRS.gov, the use of NOINDEX by the FFA members may be appropriate. 
However, it may also lead to the perception that companies are engaging in deceptive business 
practices when that may not be the case. Adding a provision to the MOU on the use or not of 
NOINDEX would provide clear guidance to the FFA members and would aid in clearing up 
these perceptions. 

4.3 Assuring Adherence to MOU 

As the sponsor of the MOU, the IRS has an obligation to ensure FFA and members abide by the 
agreed upon provisions outlined in the MOU. The Free File program office fulfills that duty by 
closely managing the program and associated relationships. The office makes certain that 
member offers are complete, free file coverage is available to all eligible taxpayers, and that both 
IRS’s and member Free File sites are ready for filing season—all in accordance with the 
specifications of the agreement. 

While the IRS has a responsibility to safeguard the provisions of Free File to taxpayers, it shares 
the duty to oversee MOU compliance with FFA. The IRS can only be an effective steward by 
working in collaboration with FFA. This section provides MITRE’s understanding and 
assessment of the overall management of the program, including a review of program objectives, 
program structure and member oversight, and compliance activities. 

4.3.1 Program Objectives 

The mission of the IRS Free File program, as noted in the IRS’s Free File Program Management 
Plan, is to “offer free tax preparation and e-file to economically disadvantaged and underserved 
populations, at no cost to the government, by partnering with the Free File Inc. (FFI) a 
consortium currently made up of 12 tax preparation software companies 62 .” 

In support of that mission, IRS-specific business objectives guide the Free File program office in 
their administration of the program. These objectives highlight the program office’s 
responsibility to provide Free File through the PPP and the Service’s secure and reliable portal: 


62 Draft Program Management Approach for Free File, IRS Free File Program Office, undated [Free File Program Management 
Plan.v3.docx.] 
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• Provide free electronic tax preparation and filing to taxpayers through a public-private 
partnership model with a focus of serving the underserved and disadvantaged 
populations. 

• Provide a safe and trusted portal via IRS.gov for taxpayers to link from IRS.gov to FFI 
software companies. Deliver web content in clear and easy to understand language. 

• Convey Free File infonnation in a clear and easy to understand language. 

As written, these objectives clearly define the what, how, and to whom the IRS is providing the 
program. 

Article 2 of the Free File MOU outlines the shared objectives for the program partnership: 

“Members shall work in concert with the IRS to increase electronic filing of tax returns, 
which includes extending the benefits of online federal tax preparation and electronic 
filing to economically disadvantaged and underserved populations at no cost to either the 
individual user or to the public treasury. ” 

These objectives were established at the start of the partnership and, while they reflect IRS’s 
mission, they include the specific goal to increase electronic filing of tax returns, as discussed in 
Section 1.1.2. They are then translated in the MOU to more specific directives for FFA members, 
which make direct reference to the two foundational, but dated, legislative mandates for the 
program (RRA and the President’s FY2003 Budget). 

Members shall also: 

• Make tax return preparation and filing easier and reduce the burden on individual 
taxpayers, particularly the economically disadvantaged and underserved populations 

• Support the IRS’s statutory goals of increased electronic filing, pursuant to the IRS 
Restructuring and Refonn Act of 1998 

• Provide greater service and access to the Services to taxpayers 

• Implement one of the proposals in the President’s Fiscal Year 2003 budget, specifically 
to encourage further growth in electronic filing by providing taxpayers the option to file 
their tax return online without charge using cooperation with, and encouraging 
competition within, the private sector 

These directives have changed little since the program was established and they reflect the 
nascent program’s focus on increasing volume of the agency’s larger e-file initiative. As noted 
previously, given the e-file rate for individual income tax returns was 88 percent in FY 2018, the 
IRS has now met and surpassed the original 80 percent goal stated for e-file. 63 This issue was the 
focus of recent external reviews of the program and both IRSAC and TAS recommended the 
development of new goals, objectives, and performance measures for the program; the TAS 
specifically recommends “targeted-use percentage” goals. 


63 IRS Filing Season Statistics for Week Ending August 31, 2018, available at: https://www.irs.gov/newsroom/filing-season- 
statistics-by-year. 
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However, when asked about their current objectives for the Free File program, IRS Submission 
Processing leadership expressed they sought to “ensure there is a free electronic filing option for 
lower- and middle-income taxpayers, and that those who are eligible for the program are aware 
of it.” It was not their goal, necessarily, to target an increase in the use of the program, but rather 
to ensure that all eligible taxpayers know that Free File is an option for the preparation and filing 
of their tax returns. 

They also discussed the need for and challenge of better defining the true demographic of 
Free File users to bring some baseline clarity to the management of the program. Historically, 
when reviewers of the program attempted to describe the Free File program’s “success,” they 
reference as a measure the annual number of Free File returns as a percentage of the total 
taxpayers eligible for the program. As noted in Section 3.3, while the total number of Free File- 
eligible taxpayers is 70 percent of all individual filers, there are many reasons taxpayers, even if 
they are aware of the option, choose not to use it (e.g., they use a paid preparer, they desire a 
faster refund, they use VITA, or they use a free commercial offering). 

When members were asked about the objectives of the program, they consistently cited the 
principles at the heart of the Article 2 MOU objectives: to provide free online tax preparation and 
e-filing to low- and middle-income taxpayers, and the underserved. While all members thought 
the original objectives were still valid, several members acknowledged that the program’s early 
aspirations were tied to increasing e-filing, which is no longer 
the primary aim. 

When asked what success for the program looked like, 
members cited two essential characteristics: that eligible 
taxpayers are aware of the program as an option—and either 
use it or make an informed choice not to—and that taxpayers 
have a positive experience with the program. Members want 
users to find the program easily accessible, simple, and straight 
forward; they strive for taxpayers to be satisfied with the 
interaction and consider using the offering in the future. 

Based on their stated goals for the IRS program office, the IRS 
is meeting their currently articulated objectives for the internal 
administration of the program. However, these goals and the shared objectives for FFA/IRS in 
the MOU, do not seem to accurately reflect, with sufficient specificity, the true intent and current 
strategy of the program today. While the shared objectives are still wholly supported by FFA 
members and leadership, they are vestiges of the program’s founding and updating them to more 
explicitly convey to both internal and external stakeholders the current program’s desired 
outcomes would benefit the program. If awareness and availability are indeed the present goals 
for the program, the IRS should use, and negotiate, more precise language that expresses those 
outcomes. 

Updated goals would enable the IRS to develop and communicate a meaningful measure of 
success for the program. Recent external reviews used the program’s current participation rate 
against all eligible taxpayers as a proxy to measure program success. Due to the nature of a 
PPP—discussed in more detail in Section 4.4—and the vagaries of taxpayer behavior, such a 
measure is not apt. 


"Americans are a hodge-podge 
of people with varying needs 
when it comes to preparing and 
filing tax returns. 

A blanket free option approach 
will never cover 100% of the 
eligible filers, and it shouldn't. 
[FF is] a great niche program 
that will never cover 100% of 
eligible taxpayers." 

-FFA member 
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IRS and FFA could update and clarify the shared goals for the program by revisiting and 
formally redefining objectives for the Free File program and outlining: what outcomes for the 
program IRS leadership see as success; what IRS operational measures would indicate progress 
and could be used to manage by; and what data is needed to help explain the impact of the 
program. If, for example, availability and awareness are the future goals for the program, 
ensuring availability would hinge on the continued effective management of the MOU and 
relationship with FFA; while increasing awareness would focus on increased outreach and 
promotion to reach all eligible Free File taxpayers. Increased participation may be a by-product 
of these activities but would not be a goal. Formally evaluating this population, as outlined in 
Section 3.2, to determine how candidate taxpayers fde and why they fde as they do would help 
the IRS better understand the true Free File population to target to meet awareness objectives. 

MITRE cautions against the addition of prescribed targets for taxpayer participation in the MOU 
objectives. Such targets would be both difficult to enforce fairly across members and a most 
unwelcome provision to negotiate with voluntary 
members of a PPP. As noted above, research into 
Free File candidate population behavior and current level 
of awareness would equip the IRS to create meaningful 
goals and inform discussions of perceived 
“underutilization.” 

4.3.2 Program Structure 

IRS’s Free File program is within the e-File Services 
branch of W&I, Customer Account Services, Submission 
Processing. To evaluate the current structure of the 
Free File program, MITRE considered existing program 
staffing, funding, and governance levels. 

Program Staffing 

The Free File Program Office is a part of the Industry 
Stakeholder Engagement and Strategy Office under e- 
File Services. It is composed of three full-time 
employees—a team lead, senior program analyst, and 
business analyst. The senior manager of the Industry Stakeholder Engagement and Strategy 
Branch divides management time among the duties of that office, which include: 

• Administering the Free File program 

• Supporting the needs of internal business organizations to interact with the largest 
external partner successfully 

• Supporting the broad tax ecosystem by enabling external partners to interact successfully 
with the IRS 64 


Stakeholders supported by IRS's Industry 
Stakeholder Engagement and Strategy Branch 

Council for Electronic Revenue 
Communication Advancement 
National Association of Computerized Tax 
Processors 

Federation of Tax Administrator's Tactical 
Advisory Group 

Largest software companies via the 
National Account Management program 
IRS Pre-Filing Season Meeting 
Reporting Agents Forum 
Software Developers Conference 
Payroll Industry Calls 
Largest Transmitters Conference Calls 
Contact Center Forum 


64 Internal Revenue Manual, 12-28-2018. 
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The host of internal and external stakeholder groups the Free File Office director supports in the 
larger of role of Industry Stakeholder Engagement and Strategy Branch Chief are included in the 
box to the right. 

The Free File Office manager and team lead both have extensive experience with the Free File 
program, numerous years with IRS, and are retirement eligible. 

As the full set of business objectives for 
the program (highlighted at right) 
illustrates, the Free File program consists 
of two components: Traditional FF (TFF) 
and Free File Fillable Forms (FFFF). 

Given the distinct nature of each of these 
components, resource requirements to 
support each are markedly different. 

For TFF, the IRS must effectively 
manage the relationship with FFA and its 
members, while supporting the 
collaborative management of oversight 
and compliance activities per the MOU. 

The Free File program office works with 
Online Services (OLS), W&I Operation 
Support Strategies and Solutions 
Research, and EPSS in its development 
and maintenance of program tools and 
IRS website requirements; however, 
taxpayer customer support is provided by 
member companies. If the IRS is advised 
of an issue within the TFF program, they 
refer the issue directly to the member 
companies to remedy. 

Two assumptions, as noted in IRS’s Program Management Approach for Free File, highlight the 
parameters of the IRS program office’s support for TFF: 

• FFA will continue to provide management oversight of member companies 

• Each Free File member that provides free interview-based software will continue to 
provide customer service 

For FFFF, in contrast, the program office is responsible for all aspects of maintenance and 
customer service support for the fillable form. Such duties include managing fonn requirements 
and changes, creating/updating an in-depth user guide, updating line-by-line instructions, 
training for W&I Accounts Management and EPPS staff, updating/testing an error search tool, 
and managing a FFFF mailbox. Responsibility for the FFFF consumes the majority of the time of 
two of the three full-time office staff. The program office team is focused on reducing errors and 
improving customer experience for the FFFF. 


The IRS s FF Program Business Objectives 
(highlighted by component) 


• Provide free electronic tax preparation and filing to 
taxpayers through a public-private partnership and 
disadvantaged populations. 

• Provide a safe and trusted portal via IRS.gov for 
taxpayers to link from IRS.gov to FFI software 
companies. Deliver web content in clear and easy to 
understand language. 


• Provide a forms-based unbranded free federal 
preparation alternative to taxpayers who don't 
gualify for the traditional FF software (Free File 
Fillable Forms). 

• Provide customer service support for the FF Fillable 
Forms utility, including self-help options. 

• Offer self-help interactive tools on IRS. gov to 
reduce taxpayer burden and offer 24X7 services. 
The tools enable a filer to self-identify corrective 
action(s) necessary to successfully process the FF 
Fillable Forms federal tax return. 

• Convey FF information in a clear and easy to 
understand language. 

Source: Free File Program Management Plan v3 
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The principal driver for program office staff is planning and preparing for, and then executing, a 
successful filing season for both TFF and FFFF. For TFF, each IRS Free File Program Office 
employee completes the formal website review of four members (between September and 
December), and works with member POCs to resolve any issues found. A high-level summary of 
the primary duties and estimated percentage of time each Free File Program Office employee 
spends on each program component is provided in Table 7. 

Table 7. Free File Program Office Key Areas of Focus. 


Employee 

Key Areas of Focus* 

Estimated Time per 
Component 

Team Lead 

Acts, in effect, as a national account manager for FFA, 
managing the agency response to FFA's industry partners 

Plans program activities 

Supports FFFF customer service 

Reviews and supports team TFF activity 

~40%TFF 

~60% FFFF 

TFF Senior Program 
Analyst 

Supports program planning activities 

Manages TTF requirements and company offers 

Ensures TFF program documents, reviews and activity are 
completed timely 

100% TFF 

FF Fillable Forms 
Business Analyst 

Supports FFFF customer service 

~85% FFFF 


Manages FFFF mailbox 

Supports TFF requirements in the Fall 

~15% TFF 

1 * Areas of focus list is not comprehensive, but rather highlights key duties; estimated time per component is based on 1 

| interviews with program staff. 



Given the nature of managing a public-private partnership, and the results evident in MITRE’s 
assessment of member compliance, including the empirical success of the program as a PPP (see 
Section 4.4), IRS’s current program resource allocation—for relationship management, planned 
periodic reviews, and direct, transparent collaboration with FFA members—is appropriate for the 
current level of work required by the Free File program. 

As noted above, the Free File program is not a traditional contractual arrangement and success 
for the program as a PPP calls for joint responsibility and decision making, where oversight and 
compliance duties are shared. Evident from interviews with both FFA and IRS stakeholders, 
much of the successful operational performance of the TFF program to date is due to the well- 
established relationships that IRS program staff, particularly the program manager and team lead, 
maintain with FFA leadership, members, and contractors. Some relationships have spanned 
decades and the level of trust, professional respect, and cooperation that exists is central to the 
program’s accomplishments. 

Given key personnel within the program office are currently retirement eligible, the IRS must 
ensure, perhaps more so than in a traditional program, that knowledge transfer and succession 
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planning are deliberate and well-planned. In addition to gaining technical expertise, new FF staff 
must have the time and space to build trust and establish collaborative working relationships. 

Moreover, any additional or more in-depth technical reviews of member websites or significant 
compliance process changes would require a careful assessment of both the nature and required 
time of the new duties to ensure the workload and required skill sets are appropriate for current 
staff resources. And, more importantly, the IRS must ensure the new duties support and do not 
jeopardize the high level of relationship management and trust that is required for such a PPP— 
and that the changes do not impact the spirit or the letter of the program. 

Program Funding 

Currently, the Free File program does not have a marketing budget. As such, the Free File 
program office works with W&I Communications and Liaison to utilize the existing 
communication and promotion vehicles available to all IRS programs, e.g., press releases before 
and during filing season and links on the IRS.gov website. As a part of IRS’s larger, annual 
campaigns, promotion of Free File in news releases, tax tips, and fact sheets is not necessarily 
specifically targeted to the Free File population. A 2018 redesign of the IRS.gov main page 
added a “File Your Taxes for Free” button at the top of the page to simplify navigation to Free 
File. There is also presently a link to Free File on page one of the Form 1040 instructions. 

IRS also employs social media to promote Free File, actively posting on Facebook, Linkedln, 
Instagram and @IRSNews and @IRSenEspanol. A comprehensive review of IRS’s recent FF 
outreach activities can be found in the taxpayer journey IRS.gov analysis in Appendix F.2 

From 2006 through 2009, the Free File program conducted a Free File Customer Satisfaction 
Survey, utilizing the firm Russell Research. This telephone survey of a nationally representative 
sampling of Free File users provided the program with insight into users’ perceptions of 
Free File and areas for program improvement. The survey provided information not available 
from IRS data routinely collected and enabled a more in-depth understanding of the Free File 
customer experience. Drivers for Free File usage and returning to the program were explored; in 
the 2009 survey, ease of navigation and likelihood of using the same provider were assessed. 
2009 was the last year the survey was conducted. 

Given paragraph 4.36 of the Free File MOU specifies that promotion of the Free File program is 
the IRS’s responsibility, the IRS has relied on unpaid vehicles and standard outreach tactics to 
fulfill this provision and increase awareness of the program. Past external reviews of the program 
recommended the IRS consider member companies assistance in promoting the program; 
however, several constraints need to be addressed to support such a pursuit, e.g., does such 
action align with the spirit of the PPP, and how would the IRS and FFA ensure such promotion 
would not result in the exclusive benefit to specific members. 

Based on MITRE’s web forensic analyses (see Section 4.2.2), paid web advertisements have a 
five-fold more effective result than organic web search hits. However, before pursuing the need 
for paid marketing and promotion, which would likely require legislative action, the IRS would 
benefit from better understanding the current level of awareness of the Free File program as an 
option. Low participation does not always indicate a lack of taxpayer awareness as other factors, 
not only cost, drive a taxpayer’s choice of how to file. The last awareness study was conducted 
in 2011 and conducting another such study would allow the IRS to baseline the current level of 
awareness and would inform strategic goal setting for the program. 
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In addition to assessing taxpayers’ current level of awareness, resurrecting the use of a taxpayer 
survey, not just for Free File users but also for those taxpayers eligible but choosing not to use 
the Free File program, could provide valuable insight into what other options taxpayers are using 
and what influences taxpayers to choose them. The results of a satisfaction survey, combined 
with demographic and migration data provided by IRS Research, can provide the specific 
targeted populations to make the Free File outreach and promotions more effective, and give a 
clearer understanding of the true population for Free File. Provision 4.28 of the MOU specifies 
that FFA members “will provide necessary support to accomplish a customer satisfaction 
survey”; hence, the IRS may request and negotiate assistance from FFA members in the 
administration of such a survey. 

If, based on future program direction, unpaid outreach continues to be the primary avenue to 
promote the program and increase awareness, the IRS could look to leverage related IRS 
programs (and other federal sites) to expand the Free File program’s reach, as they currently do 
with VITA, e.g., EITC, Low-Income Tax Clinics (LITC), other agencies (Department of 
Veterans Affairs, Department of Defense), universities, and parents of first-time filers. 

Program Governance 

In assessing the Free File program’s governance, MITRE considered both the Free File 
program’s internal monitoring to ensure the program is operating effectively and supporting 
IRS’s strategic goals, and the shared governance of the program as required by a PPP. 

Internal Program Monitoring 

Like most long-standing, established programs, the Free File Program Office provides program 
status in routine, as needed updates to IRS leadership and there is no prescribed program 
performance review. 

The office monitors and reports on the accurate and timely completion of tasks against its 
schedule to ensure a successful kick off and execution of filing season, and collects and monitors 
program data, such as weekly volume of accepted returns during FS. The office annually runs a 
demographics report of Free File returns to obtain insight into the Free File user population and 
conducts ad hoc migration analysis to assess movement of taxpayers into and out of the Free File 
program. 

Oversight and advisory boards that assess and support the program office’s performance include 
the Electronic Tax Administration Advisory Committee, IRSAC, TAS, TIGTA, and the U.S. 
Government Accountability Office (GAO). A crosswalk of MITRE recommendations with 
reports from IRSAC and TAS is included in Appendix B. 

IRS/FFA Partnership Management 

Given the Free File program is a public-private partnership and not a traditional government 
program, true governance of the program and its performance is shared between the IRS and 
FFA. As noted in the IRS’s Free File Program Management document, the program “is governed 
by W&I Submissions Processing and the FF Inc.” As such, the IRS meets with FFA leadership 
and FFA members in the spring and fall of each year to discuss status and proposed changes to 
the program. For day-to-day operations, the IRS senior manager of the Free File program 
maintains open communication and manages the relationship with the FFA executive director; 


PRE-DECISIONAL - FOR INTERNAL IRS W&l USE ONLY - NOT FOR PUBLIC RELEASE 

54 


PRE-DECISIONAL - FOR INTERNAL IRS W&l USE ONLY - NOT FOR PUBLIC RELEASE 


the IRS Free File program team members collaborate directly with FFA members to address 
routine status and items of concern throughout the year. 

As part of the management of the MOU between FFA and IRS, one year before the expiration of 
the MOU, the IRS Free File team develops a compendium of potential improvements for the 
program based on lessons learned and recommendations from oversight bodies. These 
improvements are reviewed and prioritized by IRS CAS leadership and the list is used to develop 
IRS’s negotiating options for a revised MOU. Those options are then vetted by the W&I 
Commissioner. After negotiations with FFA are complete, FFA legal counsel and IRS General 
Legal Services update the language of the MOU to reflect the renegotiation. As stated 
previously, since the program began, updates to the MOU have been additive and have not been 
part of a comprehensive review of the entire document. 

Effective program governance is based on sufficient articulation of program goals and adequate 
information on program perfonnance. Evaluating the Free File program through the lens of a 
PPP (which is outlined in detail in Section 4.4), and given the level of the program’s current 
compliance with the MOU, IRS’s existing governance of the FF program is appropriate and 
effectively allows the program to fulfill the IRS’s currently articulated goals for the program. To 
help ensure the MOU remains an unambiguous and durable document by which to manage the 
program, the IRS should encourage a periodic global review of the MOU to make certain it 
accurately and clearly reflects program objectives. 

Good governance also focuses on decision making and requires appropriate data and reporting to 
inform those decisions. Currently, the Free File program office is able to provide an appropriate 
level of infonnation on program perfonnance based on existing program goals. However, as 
noted above, to enable a more accurate discussion of the program and help the IRS, as well as 
interested stakeholders, better understand the impact of the Free File program more clearly, the 
IRS would benefit from continued research to determine the true target population of Free File 
users and eligible taxpayers’ preferences when filing (as referenced in Section 3.2). 

While gathering data on the current population of Free File participants allows for more targeted 
marketing and effective outreach to current users, if the IRS seeks to increase awareness of the 
program to all those eligible and likely to use it, analysis of the larger population that is eligible 
but not currently participating would be beneficial. Such data could help the program better 
equip IRS leadership in program advocacy and ensure the appropriate level of resources continue 
to support the partnership. More and detailed data to explain the program’s direction would 
create enhanced clarity for the public and increased stakeholder understanding of the program. 

The introduction of measures, metrics, or targets into a PPP brings with it a multitude of 
challenges. Given the nature of the IRS/FFA partnership and current objectives of the program, 
trying to measure the value and success of the program is difficult; any introduction of such 
targets would need mutual agreement on what success looks like and would need to ensure that 
all members remain free from attribution and bias, and proprietary data is respected. 

4.3.3 Member Oversight and Compliance Activities 

Consistent for a public-private partnership, Free File oversight and compliance activities are a 
shared responsibility between the IRS and FFA. The Free File MOU requires the IRS and the 
FFA ED to monitor and assess member compliance on an ongoing basis as part of program 
oversight. 
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Each year, FFA enters into an operating agreement with the FFA members. As part of the annual 
application for membership, each member must complete and sign a questionnaire that certifies 
compliance with select provisions of the MOU. This self-attestation is shared with the IRS as 
evidence of compliance. The FFA also contracts with an independent auditor to conduct a 
comprehensive audit of member sites to ensure member websites are in compliance with the 
provisions of the MOU. In addition, due to the competitive forces at work among FFA members, 
members are compelled to police each other regarding MOU compliance. Any issues they detect 
are reported to the FFA ED. 

As part of its efforts to ensure a successful filing season for the program, the IRS Free File 
program office ensures the members’ offers are timely and complete, and their software and 
website are ready for the new filing season. Employing a set of established procedures and tools 
to assess member MOU compliance, the IRS conducts website reviews, evaluates member offer 
coverage and monitors member acceptance rates. Specifically, the IRS Program Office: 

• Confirms member offers fall within the agreed upon coverage of not less than 10 
percent and not more than 50 percent of the eligible taxpayer population; the program 
office forwards a calculator tool developed by IRS Research to assist members in 
detennining this coverage and developing their Free File offers. 

• Ensures members meet a 75 percent acceptance rate for Free File returns 

throughout the filing season; an IRS data analyst generates a report of accepted Free File 
returns on a weekly basis and acceptance rate pass/fail is part of IRS’s mid-year website 
review. 

• Conducts two website reviews of each member company, one pre-filing season review 
(in December/January) and one mid-year review (March-May). The program office uses a 
website review guide, assessing each member site against 28 areas of the MOU with a 
pass/fail rating. The mid-year review contains two additional elements focused on 
acceptance rate and free extensions. 

After conducting its member website reviews, the IRS program office works collaboratively with 
the FFA independent auditor to adjudicate the findings. The FFA auditor separately evaluates all 
items included in the IRS’s website review guide, in addition to conducting a significantly more 
comprehensive audit of member sites, particularly in the area of web security, website links, 
search results, and source code. All member problems that need to be addressed are captured and 
tracked until they are remedied. The Program Office works directly with the FFA auditor and the 
member POCs to address deficiencies as part of a formal resolution process. To date, there has 
not been a case where a member issue was not adequately resolved. In addition to these fonnal 
reviews, the IRS program office works directly with members and FFA leadership to jointly 
address MOU compliance issues, if necessary, as they arise. 

Figure 12 shows the key program management and oversight activities conducted by the IRS. 
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received from W&l received from W&l 
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Confirm Member 
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Filing Season 
Member Website 
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Website Review (including 
formal Acceptance Rate 
review) 


*Dates based on FS 2019 


Figure 12. Key IRS Free File Program Management and Oversight Activities for TFF. 

Based on discussions with both the IRS and FFA representatives, and in light of the results of 
this assessment, MITRE considers the IRS’s current compliance processes adequate and 
effective to support the integrity of the program. IRS’s current oversight and compliance 
activities are the result of the partnership’s continuous assessment and improvement of the 
program as it has evolved. As evidenced in the history of the Free File Program Agreement 
outlined in Section 4.1, the IRS program office has incorporated with each iteration of the MOU 
additional or modified provisions to the compliance process to address potential deficiencies 
identified by both parties and outside bodies. For example, the formal addition of a mid-year 
review of member sites, added last year, reflects the program office’s considered response to 
external recommendations. 

As noted above, the recent suggestion for the IRS to perfonn additional or more extensive 
compliance reviews must be carefully weighed against the true return of such a provision. As a 
PPP, any threat to the integrity of the program should be solved through the collaborative work 
of both FFA and IRS. Additional member requirements negotiated on behalf of the IRS may 
reduce members’ incentives to participate and jeopardize the overall gains of a successful 
partnership. 

Finally, when the Free File program began, the program “led the charge,” as one IRS staff 
member advised, on security requirements for e-file. Hence the original MOU was populated 
with the detailed security and privacy requirements that the IRS was advancing as part of its 
larger e-file efforts. Today, however, Free File members are held to the same security 
requirements, and no more, as all e-file providers. Currently, member MOU-related security 
compliance is monitored in part through self-attestation, in part through website review, and 
more thoroughly during the FFA independent audit of members each year. The IRS receives all 
companies’ PCI scans. Given all FFA member companies are authorized IRS Electronic Return 
Originators (EROs), are subject to the assurance process of that role, and are members of the 
Security Summit, the IRS and FFA may want to consider reviewing the MOU with a critical eye 
to what security language is still relevant and warrants inclusion. To offset any risk, the program 
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may want to increase the level of corroboration with IRS’s larger e-file security assessment arm, 
e.g., to ensure they are infonned if any member is out of compliance with the e-file requirements. 

4.4 Assessment of Public-Private Partnership 

As described in Section 1.1.4, many media reports and congressional concerns about the 
Free File program have focused on the number of taxpayers who take advantage of the program. 
While it’s important to have metrics to determine whether a program is meeting its objectives, 
other factors can also determine the success or failure of a program. The Free File program is a 
unique entity that was created to provide a service to taxpayers that neither the government nor 
private industry could address alone. As depicted in Figure 13, this partnership requires a balance 
that serves the interests of the government and taxpayers, but also creates a value to the for-profit 
industry that provides the service. Without that value, private companies may decline to 
participate. As a piece of the overall assessment, MITRE looked at the partnership itself, separate 
from the service it provides. 


IRS 

Lower administrative and oversight 
cost (including, tax processing and 
efficiency costs due to increased e- 
filing; Free File administrative and 
oversight costs, etc.) 



Free File, Inc. 

Increased business development 
and revenue generation 
opportunities. 




Figure 13. Public-Private Partnership: Balance of Value to Stakeholders. 

The lack of a shared understanding of what a PPP is makes the process of evaluating whether a 
PPP has been successful complex. 65 In looking at the Free File program, MITRE uses existing 
literature to provide a framework to assess the program’s performance as a PPP. 


65 G. A. Hodge and C. Greve, "On Public-Private Partnership Performance: A Contemporary Review," Public Works 
Management & Policy, vol. 22, no. 1, pp. 55-78, 2016. 
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Research suggests using a broader set of criteria for assessing whether partnerships are 
successful. For example, other key attributes of successful PPPs including democratic 
responsiveness and process legitimacy, are considered to be equally important values for public 
undertakings. 66 

In addition to interviews with each partner and review of literature and documentation regarding 
the Free File program, MITRE used a comprehensive framework for assessment of cross-sector 
collaborations 67 to assess the Free File program PPP. The authors explain that the framework 
encompasses dimensions from the disciplines of public management, administration, and 
accountability. They suggest that PPPs can be evaluated in three categories of measures: 

1. Democratic accountability, otherwise stated as the degree of stakeholder engagement 
and support. Do both public and private leaders and partners clearly endorse and support 
the work? Are partners participating and executing on their commitments? Do all partners 
believe they are gaining the value through their support and participation? 

2. Procedural legitimacy, otherwise stated as process measures. Are processes fair, 
transparent, rational, and intentional? Process legitimacy is critical to measure because it 
improves the chances that partners, sponsors, and others accept the results that PPPs 
produce, and increases the likelihood that collaborations are managed and implemented 
responsibly and effectively. 

Substantive outcomes, which emphasizes effectiveness and efficiency, and also encompasses the attribute of 
equity (i.e., who pays and who benefits from a collaboration’s work, and what is distributed.) 

Table 8 summarizes the framework, and the attributes of value in each of the three categories 
above. The third column identifies indicators demonstrating how the Free File program measures 
against those attributes. 


Table 8. Public-Private Partnership Assessment Criteria for Free File Program. 


Attribute of Value 

Definition 

Free File Indicators 

Democratic 

Accountability 

Vertical democratic 
accountability 

Extent to which PPP is responsive 
to legal mandates, authorizers, 
funders 

• Legislators, administrative officials, 
senior agency officials show firm 
support for the program 

• MOU signed by agency 

Horizontal democratic 
accountability 

Extent to which PPP is responsive 
to partners and other external 
stakeholders 

• Implicit or explicit support from 
key stakeholders including 

Congress, TIGTA, GAO, TAS 

• Periodic MOU updates are 
responsive to concerns from 
partners and/or key stakeholders 


66 R. D. Behn, Rethinking Democratic Accountability, Washington, D.C.: The Brookings Institution, 2001. 

67 S. B. Page, M. M. Stone, J. M. Bryson and B. C. Crosby, "Public value creation by cross-sector collaborations: A framework 
and challenges of assessment," Public Administration, vol. 93, no. 3, pp. 715-732, 2015 
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Attribute of Value 

Definition 

Free File Indicators 


Procedural rationality 

Extent to which decisions are 
based on technically and 
administratively sound data, 
analysis, and planning 

• MOU is valid for a specified period 
and jointly negotiated at regular 
intervals 

• Eligibility requirements updated 
annually based on data from both 

IRS and individual members 

• In addition to periodic 
renegotiation, MOU provision 
allows for annual updates based 
on filing season data 

> 

U 

Procedural justice 

Extent to which stakeholders 

• Long standing relationships have 

E 


perceive decisions and activities 

created trust and transparency 

*+-» 

CuO 

CD 

_i 

"fD 

Z5 

"D 

CD 

u 

O 

CL 


to be fair and transparent 

between the partners 

• Mature process for negotiations, 
equal bargaining position for both 

IRS and industry 

• IRS program office works 
collaboratively with each member 
on compliance upfront 


Operational control 

Extent to which PPP uses 
requirements, budgets, and 
schedules to oversee projects and 
activities 

• IRS Program office has 
documented procedures for 
oversight of the program, 
conducts compliance checks prior 
to filing season 

• FFI contracts yearly compliance 
checks on each of the 12 members 

• Because it is a partnership, both 

IRS and FFI work collaboratively on 
any findings from either party 

l/) 

Effective performance 

Extent to which collaboration 
achieves its goals 

• Based on the four objectives 
articulated in the MOU, the parties 
achieved their goal by providing 
free filing to taxpayers and by 
increasing the e-filing participation 
to more than 90 percent 

• The goals should be updated to 
include new performance metrics 

E 

Efficient performance 

Extent to which collaboration 

• The cost of the IRS providing a free 

o 

u 


achieves its goals at reasonable 

offering itself would be in the 

+-» 

O 

CD 

> 

"■4— 1 

c 

ro 

+J 

l/) 

-Q 

LO 


costs 

hundreds of millions of dollars 68 . 

By industry providing free access, 
the IRS is able to minimize costs 
with three FTE program office 
staff. Partners receive benefits 
cited in Appendix A, The 

Economics of Free File. 


Equity of benefits 

Extent to which benefits of 
collaboration are spread 
appropriately among stakeholders 
and the public 

• IRS benefitted from not having to 
create its own e-file program, 
while industry benefited from 
keeping the IRS out of the market, 
the IRS was able to fulfill the 
President's Management Agenda 
at the time of inception and 


The MITRE Corporation, "Advancing E-file Study - Phase 2 Report," Internal Revenue Service, Washington, D.C., 2010. 
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Attribute of Value 


Definition 


Free File Indicators 


provide citizens with the ability to 
file electronically for free. 

Industry has received benefits 
(stated above) 

Because the 12 members are of 
various sizes, the 10/50 rule was 
initiated to provide equity among 
the members and prevent the two 
large companies from drawing all 
traffic 


Equity of payment 


Extent to which costs of 
collaboration are spread 
appropriately among stakeholders 
and the public 


Regardless of size, each member 
pays an equal amount to belong to 
the Alliance. While MITRE did not 
specifically ask each member 
about whether they perceived this 
to be fair, no negative comments 
were made. 

IRS maintains a small program 
office to administer the program 


Problem-solving 

capacity 


New behaviors or norms that 
increase the potential to address 
complex problems; reflects less- 
tangible outcomes that affect how 
partners do their work now and in 
the future 


IRS maintains a strong working 
relationship and environment of 
trust with the members 
As technology and the 
environment change each year, 
the Alliance and the IRS work 
collaboratively to address these 
with changes to the MOU 
The MOU provides opportunities 
for innovation within the industry 


Using the above framework, MITRE concludes that the Free File program is an effective PPP 
given the value that the Free File program provides to the public as a venue for taxpayers to 
prepare and electronically file their tax returns for free; to the IRS in providing that venue as a 
government service; and to the industry, not only in terms of preserving their unique role of 
preparing and filing taxes, but also in providing a potential economic benefit to members as 
discussed in Appendix A. 

4.5 MOU Renewal in 2021 

While the IRS could use the MOU to negotiate stricter terms to alleviate taxpayer and public 
concerns about FFA activities, there are limitations to this process. First, the MOU is ultimately a 
two-way negotiation tool. Moreover, changing MOU provisions that restrict or dictate FFA’s 
business practices may require other concessions to FFA members in return. Individual FFA 
members may demand concessions that align with the members’ own economic and business 
incentives adding another layer of complexity to the negotiation process. 

Substantial changes to the MOU that either aim to ease or tighten restrictions on providers of 
Free File can influence both the use and provision of Free File services. Easing restrictions on 
FFI members would create opportunities of extracting additional revenue from Free File users 
through commercial tactics. If providers impose commercial tactics on Free Filers, taxpayers 
would be indifferent between Free File and comparable commercial software, potentially 
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reducing the use of Free File. This would be a considerable problem at high volumes of Free File 
usage, because a large drop in the demand for Free File could affect overall e-filing numbers. In 
the current state of affairs, this is not a significant concern as a drop in the current Free File 
numbers would not affect overall e-filing figures enough to cause it to fall below the 80 percent 
overall e-filing target of the IRS. 



Figure 14. The MOU as a Negotiation Tool. 

Figure 14 demonstrates how the provisions in the MOU can impact the provision of Free File 
services. The number of firms that are willing to participate in the MOU depend on the extent to 
which the IRS imposes restrictions on the business activities of tax software companies. Under 
limited restrictions, firms have greater opportunities of extracting profits from taxpayers through 
commercial business tactics. As restrictions on the FFA lessen, the Free File market approaches 
characteristics of a more competitive industry with many potential service providers and 
increased consumer choice. 

Conversely, a restrictive MOU that impedes on business development and profit-making 
activities of tax software companies would drive out firms from participating in Free File. As 
more and more suppliers leave the program the market would approach characteristics of a 
regulated monopoly in which both the number of providers as well as consumer choice is 
limited. A lower number of providers may also impact the number of taxpayers that are eligible 
to participate in the program under the limited eligibility criteria of a fewer firms. 

The discussion above demonstrates that the IRS stands in a unique position of having to balance 
taxpayer expectations from the program with the industry members’ outlook for economic gains. 
From some stakeholders’ perspective, one Free File eligible customer paying for tax preparation 
services is one too many. From FFA’s perspective, the provision of Free File member offerings 
costs money, and the pub lie-private partnership must leave room for recovering these costs and 
potentially generate additional business. In other words, the cost of provision of the free services 
to some customers must be subsidized by other paying customers. 


PRE-DECISIONAL - FOR INTERNAL IRS W&l USE ONLY - NOT FOR PUBLIC RELEASE 

62 













PRE-DECISIONAL - FOR INTERNAL IRS W&l USE ONLY - NOT FOR PUBLIC RELEASE 


5.0 Taxpayer Experience with Free File 


Taxpayers are, of course, the target population for the Free File program and its accompanying 
systems. As such, examining the taxpayer experience is crucial to a holistic assessment of Free 
File. MITRE approached the taxpayer experience from two complementary research 
perspectives: the expert perspective, represented by the heuristic review; and the taxpayer 
perspective, which MITRE investigated via usability testing interviews with prior Free Filers. 

In many cases, the findings from the heuristic review and the usability testing mirror each other; 
a major advantage of conducting these research streams in tandem is the ability to observe a 
large set of potential usability issues for a wide range of systems in the heuristic review, and 
then observe whether or not these potential issues manifest in testing with real end users 
(taxpayers, in this case). Integrating the results from the taxpayer journey heuristic review and 
the usability testing with taxpayers, the following high-level findings stand out: 

• Messaging and functionality on IRS.gov pages and within member systems to help 
taxpayers determine eligibility and ineligibility for Free File needs improvement 

• Taxpayers lack understanding of what Free File is and the IRS’s relationship to the 
program; use of consistent, plain language on IRS and member pages would help 
mitigate this issue 

• Taxpayers experience difficulty navigating the Free File landing page and the 
software offers page; they lack clear “calls to action” and are too text-heavy 

• The lookup tool and the Free File member systems could take more actions to 
prevent users from making errors, and help them correct errors where committed 


MITRE conducted two research activities in order to understand how taxpayers interact with and 
experience the Free File system and to evaluate the usability of the Free File system components. 
MITRE evaluated Free File experience aspects including searching for Free File online, 
navigating IRS.gov web pages, selecting a Free File software offer, and filing a return with a 
Free File member system. 

MITRE approached the taxpayer experience from two complementary, yet distinct, angles: 

1. Taxpayer journey—expert review of Free File member systems and IRS.gov 
(heuristic review). In this activity, MITRE human factors engineers drew on established 
industry best practices and expert knowledge of system usability to evaluate various 
dimensions of usability and compliance with the MOU. MITRE identified potential 
usability issues in a systematic, exhaustive fashion and made recommendations on the 
basis of technical expertise. 

2. Usability testing sessions with Free File taxpayers. In this activity, MITRE researchers 
conducted in-person interviews with taxpayers who filed using Free File in TY 2018. 

This activity allowed MITRE to observe usability issues encountered by real end users 
and to collect qualitative data on taxpayers’ perspectives, opinions, and why aspects of 
the system caused difficulty. 
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This two-pronged approach to understanding the taxpayer experience allowed MITRE to 
mitigate some of the limitations of each individual approach. Although the usability testing has 
the advantage of greater external validity—the ability to generalize beyond the testing 
situation—due to its focus on actual end users, usability testing is time-consuming and resource¬ 
intensive, and as a result, the MITRE interviews tested three of 12 Free File member systems. In 
contrast, the heuristic review encompassed all 12 member systems. Additionally, combining the 
heuristic review with usability testing presents the opportunity to observe which potential 
usability issues identified in the heuristic review caused problems for taxpayers in interviews. 

In the following sections, MITRE reports the key findings from the heuristic review and the key 
findings from the usability testing. Connections between the two sets of findings, with 
conclusions, are spelled out in Section 5.3. 

5.1 Taxpayer Journey 

This section of the report summarizes the findings from the expert usability review of the 12 Free 
File member systems as well as ofIRS.gov Free File pages (landing page, offer page, etc.). 

In this review, MITRE human factors engineers applied three sets of criteria, known as 
heuristics, to evaluate the usability of the systems. MITRE used three fictitious taxpayer 
scenarios to navigate the Free File systems in the course of the review. The three types of 
heuristics MITRE used to evaluate the member sites and IRS.gov’s Free File-related pages were: 

1. Nielsen’s 10 usability heuristics. Factors used to evaluate overall system usability and 
user experience. These factors were rolled up to detennine a single “user experience” 
score. These heuristics are detailed in Section 2.5. 

2. MOU UX requirements. MITRE viewed the MOU requirements from a usability lens. 
MITRE derived a set of criteria based on the MOU requirements that are related to user 
experience, such as “transparency around ineligibility.” MITRE then rated the systems 
against this list of criteria and rolled up these factors to detennine a single UX MOU 
adherence score. Note, this set of MOU-related heuristics is separate from the MOU 
compliance assessment activities and should not be interpreted as evidence of member 
compliance or noncompliance with the MOU. The heuristics are also detailed in Section 
2.5. 

3. Mobile optimization. MITRE evaluated to what extent the member systems were 
optimized for use on mobile devices. 

5.1.1 Heuristic Review of Member Free File Websites 

MITRE evaluated the member sites against Nielsen’s 10 usability heuristics to derive the user 
experience score, against the MOU-related heuristics to derive the UX MOU adherence score, 
and evaluated mobile optimization to derive a mobile experience score. The severity ratings 
MITRE used to determine these scores are displayed in Figure 15. All of the scores from the 
heuristic review are rolled up into the summary-level dashboard shown in Figure 16. The 
dashboard provides a quick view of the summary scores for user experience, mobile experience, 
and MOU adherence from a UX perspective. 
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Severity Level and Description 


No observed usability problems related to this heuristic 
Nuisance problems identified 
Minor usability problems 
Major usability problems 

Severe usability problems (possible user failure) 


Note:Too manynuisance severity issues can be as bad as Minoror Major severity. 


Figure 15. Severity Level and Description. 


The information that follows provides a more detailed breakout of the scores from the heuristic 
review by each heuristic and grouping of MOU requirements. The numbers across the columns 
identify the member systems; the heuristics sets are listed on the rows. 


Dimension 

User Experience 
UX MOU Adherence 
Mobile Experience 



10 


11 


12 


tf44t4t4t4t 
* * * * * * * * t #|t 
f444t^44tt4 


SUMMARY 

At a glance, the Free File providers adhere 
to many UX best practices. Overall, they 
could improvethe most in “Error 
Prevention” and “Match Between System & 
Real World” heuristics. The most significant 
UXMOU improvement to make across the 
board is “Transparency around ineligibility.” 


Score range 


85% to 100%. Area of strength. 

^ 70% to 84%. Room for improvement, 

p 0% to 69%. Area ofweakness. 


Figure 16. Heuristic Evaluation Dashboard Summary. 

As shown in Figure 17, at a glance, the Free File member organizations adhere to many UX best 
practices. Overall, they could improve the most in the areas of “error prevention” and “match 
between system and real world” heuristics. The most significant UX MOU improvement to make 
across the board is “transparency around ineligibility.” 
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Nielsen’s 10 Usability Heuristics 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

11 

12 1 

Visibility of system status 

5 

5 

5 

5 

4 

5 

4 

5 

4 

5 

3 

3 

Match between system and the real world 

4 

5 

5 

2 

3 

4 

4 

4 

5 

5 

3 

3 

User control and freedom 

5 

5 

5 

5 

5 

5 

4 

2 

5 

5 

4 

5 

Consistency and standards 

5 

5 

5 

5 

5 

5 

4 

5 

3 

5 

3 

5 

Error prevention 

5 

5 

3 

5 

4 

4 

1 

3 

2 

5 

4 

3 

Recognition ratherthan recall 

5 

5 

5 

3 

3 

3 

4 

5 

3 

5 

3 

5 

Flexibility and efficiency of use 

5 

5 

5 

4 

3 

5 

4 

5 

2 

5 

4 

5 

Aesthetic and minimalist design 

5 

5 

5 

2 

2 

5 

4 

5 

3 

5 

5 

4 

Hep users recognize, diagnose, and 
recover from errors 

5 

5 

5 

4 

1 

5 

3 

5 

5 

5 

5 

5 

Help and documentation 

5 

5 

5 

2 

3 

4 

4 

4 

3 

5 

4 

5 

Total 

49 

50 

48 

37 

34 

45 

36 

43 

35 

50 

38 

43 

User Experience Score 

98% 

100 

% 

96% 

74% 

66% 

90% 

72% 

86% 

70% 

100 

% 

76% 

86% 


UX MOU Requirements 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

li 

12 

Privacy, security, free from marketing 

4 

4 

4 

5 

5 

4 

4 

3 

3 

4 

5 

4 

Understands offering available 

4 

5 

5 

4 

3 

4 

3 

4 

4 

4 

4 

4 

Usercontroland access 

5 

5 

5 

5 

2 

5 

3 

2 

5 

5 

3 

5 

Meets expectations 

4 

5 

5 

4 

4 

4 

5 

4 

5 

5 

4 

4 

New technologies 

4 

5 

5 

4 

5 

4 

4 

4 

4 

5 

4 

4 

Transparency around ineligibility 

4 

4 

1 

4 

4 

4 

3 

3 

4 

5 

5 

5 

Total 

25 

28 

25 

26 

23 

25 

22 

20 

25 

28 

25 

26 

MOU Score 

83% 

93% 

83% 

87% 

77% 

83% 

73% 

67% 

83% 

93% 

83% 

87% 


Mobile Experience 

1 

2 

3 

4 

5 

6 

7 

8 

9 

10 

li 

12 

Mobile optimized site 


Mobile Experience Score 

100 

% 

100 

% 

80% 

80% 

80% 

100 

% 

60% 

80% 

60% 

100 

% 

100 

% 

40% 


Figure 17. Heuristic Evaluation Dashboard. 
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The complete set of results from the heuristic assessment for individual Free File member 
websites is found in Appendix F.l. 

5.1.2 Heuristic Review of IRS.gov 


In addition to assessing member’s Free File sites, MITRE also reviewed the user experience for 
Free File-related pages on IRS.gov and other relevant IRS materials. 

Specifically, in the heuristic review ofIRS.gov, MITRE evaluated the user experience of 
determining how to file taxes for free and identify a company to use within the Free File 
program, including: 1) issues observed in web searches for Free File and related terms; 2) 

IRS.gov internal search functionality; and 3) IRS.gov navigation. MITRE identified and 
described potential user pain points along the overall experience. 

Similar to the heuristics review of the member sites, the taxpayer journey from search to lookup 
tool was evaluated against Nielsen’s 10 usability heuristics to derive a “user experience” score 
for search, navigation, and the IRS Free File landing page and application. MITRE also provided 
comments on examples of marketing materials shared by the IRS but were not included as part of 
the heuristic review due to messaging appearing in familiar user interfaces, such as Twitter or 
Facebook. 


The majority of heuristics issues were on the IRS Free File landing page and application. For 
example, the landing and software offers pages are text-heavy, difficult to scan, use unfamiliar 
tax jargon, and do not clearly present the next step to users. Figure 18 provides a snapshot of the 
IRS.gov review, specific to the IRS Free File landing page. 


First Impression of Free File Landing Page 

Above the fold view of I RS.gov Free File landing page from a Google Search, August 2019 




Navigation to Free File Landing Page 

Key metrics related to the Free File landing page 1/1/19 -4/30/19 



Figure 18. Samples from Heuristics Review for IRS.gov Landing Page. 
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Similarly, the software offers page uses overly technical or potentially unfamiliar terms like 
“lookup tool.” The list of vendors and their criteria is hard to scan, as users must read 
inconsistently formatted bullets with conditional statements that are difficult to understand. 
Issues such as these identified in the heuristic review were used to develop user interface design 
recommendations. Figure 19 and Figure 20 offer two potential design alternatives intended to 
improve scanning for FFA member offers. 



Figure 19. Design Alternative #1 to Improve Scanning for FFA Member Offers. 
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Figure 20. Design Alternative #2 to Improve Scanning for FFA Member Offers. 

Other potential opportunities identified through the taxpayer journey analysis include: 

• Free File awareness. Free File is referred to as a “ kn own” program throughout the 
messaging in the promotional materials. Given that so few taxpayers participate in the 
program, it’s likely the awareness and understanding reading “Free File” would be low for 
the average taxpayer. Explain what the Free File program is to people who have never 
heard of it in very plain and simple language. Also, gather foundational knowledge in 
support of further testing that could be done related to the value of name recognition on 
the IRS website. Users rely on finding what they need based on terms they know (e.g. 
Form 1040). In general, the Free File program is never clearly explained with its purpose, 
requirements, and how to participate. The way it’s referred to throughout the site makes it 
seem as though it’s very well advertised and explained, making users feel they may have 
missed something and fumble through the action buttons presented to them without much 
purpose. 


PRE-DECISIONAL - FOR INTERNAL IRS W&l USE ONLY - NOT FOR PUBLIC RELEASE 

69 





































PRE-DECISIONAL - FOR INTERNAL IRS W&l USE ONLY - NOT FOR PUBLIC RELEASE 


• Keep language easy to understand. Words like “software,” “e-file,” “AGI,” and “EITC” 
are used repeatedly in relation to Free File when those words may not mean anything to 
the average user. Avoid compliance-oriented language in favor of service-oriented 
language by providing more plain language descriptions of tax terms and IRS programs. 
Using the word “software” is misleading and outdated for users since it isn’t something 
they need to install. Given that the programs are accessible from mobile devices, 
recommend using words like “online” or “website” or “program.” 

• Rethink the program with the novice in mind. Determining eligibility for the Free File 
program may be intimidating to taxpayers. The vendor requirements are complex and the 
lookup tool asks for information that is not common terminology. Where possible, attempt 
to reduce the complexity of the vendor requirements for the program. 

• Keep users within the task at hand. The lookup tool assumes taxpayers are 
knowledgeable of their adjusted gross income and eligibility for the Earned Income Tax 
Credit. For novice taxpayers, the help text is not written in plain language, and references 
content and calculators outside of the tool. This forces users to leave their current task at 
hand and work on what might feel like an unrelated task, increasing the chance they do not 
return to participate in the Free File program. If an eligibility requirement includes 
knowledge of a program or calculation that is not commonly known, provide a simple 
calculator within the tool itself. 

• Clearly outline Free File options upfront. Determine the primary business goal of the 
page(s) and use it to drive layout and content development. For example: If the primary 
goal is to move people towards exploring their eligibility and the options available, use a 
task-oriented approach. Provide “Income below” and “Income above” boxes above the 
fold, with buttons that clearly describe the next step in the process. The landing page 
buries the actual “start now” process behind many clicks. Improve the overall readability 
by developing content from “bite, to snack, to meal” 69 that supports the primary business 
goal of the page. Ensure that the primary conclusions of the content are summarized 
within titles, headings, and initial content. Make sure distinct topics are separated on the 
page (one sentence paragraphs are fine) and aim to use keywords (like state, income, 
military) within the first few words of a sentence. 

• Branding strategy. The IRS marketing, public release, and social media presence around 
the Free File program does not have a consistent voice, purpose, or strategy. The design of 
the visuals connected to the various fonns of social media all follow different styles and 
themes, making it difficult to thread together a coherent brand for a taxpayer. The IRS 
loses credibility by disorienting the user with different fonts, pages, and calls to action. 
Recommend creating an overarching communications strategy around Free File that 
matches the style of the rest of IRS branding with specific calls-to-action and educational 
materials to teach those unfamiliar with the program about how to participate. 

• Continuous improvement. When detennining the primary business goal of the Free File 
page(s), capture baseline metrics from web analytics, usability testing, and overall 
satisfaction scores. For example, in Appendix F.2 (page 20) shows that only 32 percent of 


69 C. R. J. S. K. S. K. Jarrett, "Design to Read: Guidelines for People Who Do Not Read Easily.," 2010. [Online]. Available: 
http://uxpamagazine.org/people_who_do_not_read_easily/. [Accessed 15 August 2019] 
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users click through to the application from the Free File landing page, in comparison to 76 
percent from the refunds landing page to the application. While it is not a direct 
comparison (a very different audience and goal), it poses two questions: are the right users 
navigating to the landing page, and can the number of eligible users who navigate to the 
application increase (by say, content and user interface updates)? Use captured baseline 
metrics to compare the performance of future redesigns against program goals. 

• Error prevention. Help users through the basic steps of detennining an estimated AGI 
and eligibility for EITC to prevent errors. For age include more descriptive labeling, or 
update the question so it takes less thought (birthday versus age). Include a link to fillable 
forms and back to the offers page. Incorporate primary site navigation such as the new 
withholding tool. 

• Support search. Identify new ways to help users pick IRS Free File out of the sea of 
“free” offers in the most prominent search engines. Consider and test all available options: 
from paid search advertisements to more task-oriented headers that include IRS within the 
first one or two words (so that users can scan and immediately identify with IRS). To 
improve search within IRS.gov, use a wider variety of search terms. 

• Develop a design challenge to improve offers list. The list of vendors is difficult to scan 
for eligibility criteria. If the eligibility requirements remain complex, consider exploring 
alternative designs. See Appendix F.2 (page 42-43). Continue to conduct usability testing 
to ensure it is easily understood and scannable. 

The complete IRS.gov Free File assessment is found in Appendix F.2. 

5.2 Taxpayer Experience Usability Findings 

In the Taxpayer Experience Testing Report (Appendix G), MITRE reported the findings from the 
taxpayer usability testing study. In July 2019, MITRE conducted 29 in-person testing sessions 
with taxpayers identified in CDW as having used Free File to file their federal returns in TY 
2018. Taxpayers were randomly selected to receive invitation letters to participate in the study 
from a cluster of six Chicago zip codes. 

During the usability interviews, taxpayers completed three usability tasks while a MITRE 
researcher observed them. The three tasks were: 1) locate where to file a federal tax return for 
free via a web browser; 2) navigate the Free File landing page and select a Free File member 
offer, including with the software lookup tool; 3) file a tax return using one of three randomly 
selected Free File systems. For all activities, participants used fictitious tax scenarios to protect 
their privacy. Taxpayers also answered interview questions regarding their previous experience 
using Free File and their perceptions of the program. 

MITRE identified and reported all usability issues observed by the research team with each of 
the three Free File usability tasks and provided suggested solutions for addressing each of these 
issues individually. Drawing on the data collected from the usability tasks, analysis of qualitative 
data gathered during debriefing and with semi-structured interview questions, the satisfaction 
and user experience questionnaires, and metrics on taxpayer behaviors and selections during the 
tasks, MITRE identified the high-level findings discussed in the following sections. 
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5.2.1 General Free File Program Findings and Usability Issues 

Taxpayers lack understanding of what the Free File program is and its relationship to the 
IRS. 

Despite the fact that MITRE specifically recruited taxpayers recorded in CDW as having 
previously used Free File, many participants conflated the Free File program with other forms of 
filing taxes for free, including free commercial software, and were unsure whether they had used 
Free File or a free commercial version in the past. Some reported not having heard of Free File 
until the past year. Taxpayers also seemed unclear on the relationship between the IRS and 
Free File. A few never sought out IRS.gov and did not seem to realize that they should use IRS 
resources to find the program, whereas others expressed surprise that Free File offers were all 
provided by third-party vendors and not directly from the IRS. Evidence for these findings 
include: 

Usability Issues 

• Seven taxpayers expressed surprise that Free File vendors are commercial companies, not 
directly offered through the IRS 

• Taxpayers mistook finding an e-file provider for finding a Free File software vendor 

Qualitative/Debriefing Support 

• Four taxpayers conflated free versions of FFA member software offerings with Free File 
during debriefing interviews 

• Five taxpayers reported not having heard about Free File before the prior year 

• Three taxpayers suggested the IRS advertise Free File more 

Eligibility Screening 

• Of 36 taxpayers deemed eligible to participate in the study, 28 percent expressed 
uncertainty as to whether they used Free File or a free version of commercial software 

Taxpayer Pathways/Metrics 

• Four taxpayers ended their Free File search on a commercial vendor website and never 
sought IRS.gov, suggesting that they did not realize that Free File offers should be 
accessed via IRS.gov. MITRE’s study design (e.g., recruitment letters and interview 
procedures) may have raised awareness among participants about Free File and the IRS’s 
involvement. The majority of search keywords used to find Free File in the interviews 
involved some combination of “IRS” and “free.” Keywords with this combination tended 
to lead taxpayers successfully to the IRS Free File landing page 

5.2.2 Locating the Free File Program Online 

Taxpayers struggled to find the Free File landing page, including through IRS.gov. 

Most taxpayers approached finding Free File by starting with a search engine query, using 
keywords such as “IRS,” “free tax filing,” and “free file.” This suggests that the IRS needs to 
plan for taxpayers to enter the Free File system from search query results, rather than from the 
IRS home page. Some ended up on commercial vendor websites; most were able to eventually 
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find the Free File landing page on their own, though through various routes that caused 
confusion. Some taxpayers meandered through pages of IRS.gov looking for Free File and 
expressed frustration that they could not find it. 

Usability Issues 

• Three taxpayers navigated to various IRS pages while attempting to find Free File. They 
thought that Free File was hard to find, were not sure how they arrived at Free File once 
they did, or never found it on their own at all. 

• One taxpayer went to multiple pages on the IRS website including the free tillable forms, 
find a tax professional, and the forms and instructions pages. This taxpayer did not find 
the Free File page on their own. 

• One taxpayer made it to the IRS Free File landing page but then had trouble proceeding to 
the list of Free File software offers. The taxpayer veered away from the IRS Free File 
landing page. They navigated from the IRS Free File landing page to the filing for 
individuals page. 

• One taxpayer navigated to the authorized e-file provider page. This taxpayer initially 
found the Free File landing page but then navigated to the page with authorized e-file 
providers. 

Taxpayer Pathways/Metrics 

• Twenty-four participants were able to find the IRS Free File landing page 

• Of the 24 participants that ultimately found the IRS Free File landing page independently, 
10 followed suboptimal paths, meaning that they were frustrated or confused along the 
way and visited other, unnecessary pages in the course of their search 

• Twenty-six taxpayers started their search for Free File with an internet search engine 
query (e.g., free tax filing) 

o The search terms that most frequently led to the IRS Free File landing page included 
both “IRS” and “Free File” 

5.2.3 Selecting a Free File Software Offer, Including with the Software Lookup Tool 

Taxpayers experienced difficulties and confusion with selecting a software offering, 
particularly when evaluating Earned Income Tax Credit (EITC) eligibility. 

EITC eligibility requirements were confusing for taxpayers and a substantial proportion (about 
one-third) committed errors when selecting a vendor using the lookup tool due to erroneously 
answering the EITC eligibility question. Information available to taxpayers on IRS.gov regarding 
EITC eligibility largely did not resolve the problem. 

The lookup tool has room for improvements in usability. Common issues included taxpayers not 
noticing the lookup tool, not understanding its purpose, and committing errors on EITC 
eligibility. 

Errors on the lookup tool result in vendors returned that taxpayers may not be eligible for; 
taxpayers might only discover this down the road when they are midway through completing a 
tax return. Incorrectly selecting yes for EITC eligibility in particular causes more results returned 
from the lookup tool than are accurate. Evidence for these finding include: 
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Usability Issues 

• Eight taxpayers selected “yes” on the EITC question in the lookup tool despite being 
unsure of answer. 

• Five taxpayers believed that the lookup tool would verify their responses, especially on 
EITC eligibility. 

• Six taxpayers misunderstood the infonnation in the EITC modal popup. 

• Two taxpayers were confused after using EITC assistant tool. 

• Three taxpayers had preconceived notions regarding eligibility criteria and committed 
errors as a result. 

• Two did not know how to answer the EITC question and wanted to leave it bla nk or leave 
the lookup tool. 

Taxpayer Pathways/Metrics 

• Twenty-nine percent of taxpayers committed an error on the software lookup tool; 100 
percent of these errors were due to incorrect response on the EITC eligibility question 


Taxpayers were overwhelmed by the Free File software options and by the information 
provided along with options on IRS.gov Free File-related pages (e.g., Free File landing page 
and Free File software offers page). 

They were overwhelmed by the amount of information they saw when they reached the Free File 
software offers page, including the paragraphs above the Free File software lookup tool box and 
the Free File software offers list below the lookup tool box. Taxpayers also found that the lookup 
tool returned too many options. Taxpayers found it difficult to select an offer due to the number 
of choices and amount of infonnation presented to them. Evidence for these findings include: 

Usability Issues 

• Five taxpayers were overwhelmed by the amount of information on the Free File 
software offers page, particularly the Free File software offers. 

• Three taxpayers perceived too many options returned from the lookup tool and had 
trouble choosing an offer. 

• Five taxpayers did not understand if they needed to meet all or only some of the 
eligibility criteria on the Free File software offers list page in order to qualify. They did 
not understand if the criteria were “and” or “or” statements. This confusion resulted in 
taxpayers not knowing which Free File software offers they were eligible for. Some 
understood Free File software offers as being ONFY for military or EITC. 

Qualitative/Debriefing Support 

• Five participants felt the biggest difficulty was that there were too many options to choose 
from on the IRS website 
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o “There are too many choices. If there are ten there and four qualify, then it’s like eenie 
meenie miney mo.” 

The majority of taxpayers reported reasons other than eligibility criteria for selecting Free 
File software offers, in part because the eligibility criteria were long and complex. 

Taxpayers rely on familiarity with company names, particularly the large software providers, and 
previous experience with tax software. When in doubt, taxpayers drew on heuristic devices to 
help themselves establish the legitimacy of software providers, such as keywords in software 
providers’ names. In part, these strategies were necessitated by the sense that there were too 
many options and it was difficult to distinguish between them. Some taxpayers mistook 
commercial providers with similar keywords as legitimate IRS Free File program selections. 
Evidence for these findings include: 

Usability Issues 

• Taxpayers were overwhelmed by the number of options, both on the software offers page 
and in the lookup tool results. 

• Taxpayers expressed a desire to see the eligibility requirements repeated on the lookup 
tool results list, to help them choose from the list of Free File software offers. 

Qualitative/Debriefing Support 

• The majority of taxpayers cited reasons other than eligibility criteria for choosing a 
vendor. Prior experience and name recognition were the top two reasons, with a total of 45 
mentions. 

• Eligibility criteria were mentioned as the reason for selection 13 times. 

Taxpayer Pathways/Metrics 

• Nine taxpayers navigated to a commercial website as part of the process while they were 
searching for Free File. Four of the nine ultimately stayed on a commercial page, not 
realizing that they had not selected a Free File offer. 

5.2.4 Completing a Tax Return Using a Specific Free File Offer 

General usability issues across multiple software providers included finding where to enter 
student loan interest information, selecting a filing status, and navigating through the filing 
systems. 

Taxpayers were not sure where to enter student loan information and often found it where they 
were not expecting. Taxpayers had a hard time deciding between selecting head of household or 
single. They did not understand the flow, or sequence of pages and infonnation, of various 
aspects of the tax preparation systems. Evidence for these findings include: 

Usability Issues 

• Seven taxpayers were not sure where to enter their student loan information from Fonn 
1098-E while filing a return using two out of the three Free File software offers. 

PRE-DECISIONAL - FOR INTERNAL IRS W&l USE ONLY - NOT FOR PUBLIC RELEASE 

75 



PRE-DECISIONAL - FOR INTERNAL IRS W&l USE ONLY - NOT FOR PUBLIC RELEASE 


• Six taxpayers were uncertain or confused about either selecting head of household or 
single as their filing status. Taxpayers expected more infonnation within the system to 
help them make the correct choice. Instead, they had to turn to an external source for 
more information. 

o If they chose head of household, some taxpayers expected to see additional 
questions or prompts to confirm their selection but did not receive them. 

o System prompts asking taxpayers if they wanted to change their filing status were 
confusing and undermined taxpayer confidence 

• Three taxpayers did not understand the path or flow of information presented to them 
while filing. They lost track of where they were in the filing process, had trouble 
navigating menus, or did not see the pages they were expecting to see. 

• Three taxpayers were confused about what to do once they chose the quick file option. 
They were not familiar enough with tax preparation to enter each of the relevant forms 
they had for filing. 


Overall, taxpayers rated the usability of the Free File system a “B.” The filing task was the 
most difficult for participants, which is not surprising given that completing a tax return is 
a more complex task than browsing the internet or navigating IRS.gov. 

Usability scores varied by Free File software offer. There was some evidence that usability 
scores varied by age and education level, though these results should be interpreted with caution 
given the small sample size. Evidence for these findings include: 

Questionnaire Scores/Analysis 

• Participants rated task 3 as harder than tasks 1 and 2; in line with task 3 being inherently 
more complex than tasks 1 and 2. 

• Participants’ ratings of the Free File system overall varied by which Free File software 
offer they used to file a return, suggesting that the software experience colored taxpayers’ 
perceptions of the entire experience, including IRS site interactions. 

• Patterns in usability ratings by education were unclear: 

o High school-equivalency group had lowest per-task ease for Tasks 1 and 2 
(SEQ) but highest overall usability (UMUX) 

• Youngest age tercile (20-35 years) reported the best usability across entire experience, but 
there was not a conclusive age group pattern. 

5.3 Integrated Look at Taxpayer Experience 

MITRE conducted two distinct research activities to evaluate and understand the taxpayer 
experience: the taxpayer journey heuristic review ofIRS.gov Free File pages and all 12 current 
Free File member systems, and usability testing interviews with 29 prior Free File users. 
Identifying themes and integration points between these two activities highlights crucial areas for 
improvement and recommendations moving forward. 
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Improve functionality and messaging around taxpayer eligibility and ineligibility for Free 
File, on both IRS.gov pages and within member systems. 

Free File systems are mandated by the MOU to clearly communicate ineligibility for Free File; 
additionally, selecting an offer for which they are eligible is one of the core tasks that a taxpayer 
must perfonn in order to access Free File program services. As such, it is of central importance. 
Findings from both the heuristic review and taxpayer usability testing show that this is an area 
for improvement for IRS.gov and member sites. The heuristic review showed that the majority of 
the 12 Free File member systems had severity ratings suggesting potential usability issues around 
users not being able to quickly recognize ineligibility for Free File on a particular system. In the 
taxpayer testing, MITRE observed that many taxpayers struggled to determine eligibility at the 
outset; specifically, participants had trouble answering the EITC eligibility question used in the 
software lookup tool and were confused by the many eligibility criteria presented on the software 
offers page. Taxpayers largely relied on prior experience with vendors and name recognition to 
select an offer. Taken together, there is risk that taxpayers will mistakenly select a vendor for 
which they are not eligible, and not realize it until late in the return filing process. 

Promote understanding of what the Free File program is throughout IRS.gov by using 
consistent language and labels. 

The usability testing with taxpayers revealed two issues that affected their ability to effectively 
locate and use Free File: they had difficulty navigating the internet and IRS.gov to find Free File, 
and they made comments during the interviews that reflected a misunderstanding of what the 
Free File program is (e.g., expressing surprise that the IRS doesn’t have their own tax filing 
software, or assuming they should go straight to a commercial website rather than accessing Free 
File through the IRS). In parallel, the heuristic review identified similar-sounding links and 
buttons throughout IRS.gov that taxpayers might try to follow to find Free File if they are not 
familiar with the program (e.g., File > How to File). 

In the usability study, MITRE observed multiple taxpayers follow various links in their pursuit 
of Free File, often getting frustrated. Figure 21 shows the sub-optimal paths that 10 taxpayers 
interviewed took to reach the Free File landing page, including accidentally landing on a 
commercial site before finding IRS.gov, and reaching the Free File landing page but accidentally 
navigating away from it by using various links other than the correct “Start Free File Now” 
button. Additionally, MITRE observed that participants most successful in identifying Free File 
quickly from a web browser search used a combination of terms that include “IRS” and “free 
file” or “file taxes for free,” rather than relying only on “IRS” or “file taxes for free.” Together, 
these findings point to the need to consistently brand Free File, use consistent tenninology, and 
raise awareness of this terminology. These steps should help taxpayers more quickly and easily 
home in on the information they need. 
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Task 1 Acceptable-Suboptimal Pathways 
(n = 10) 


Types in "US IRS" 


Google search 



Task 1 Path 1b 




Task 1 Path 2a 


Types in "free tax filing" 


© 


P28 selected e- file.com because it was 
the first result on the list. They navigated 
to the create your account page before 
returning to Google. 


'This is very confusing, I'm not sure how I got 
, © © \ here...This [Free File link in menu, and the Free 
.— J File button on the page] is under ’popular'? 

Seems to me it should be more prominent." P5 


Lands on 
IRS.gov 
home 


Navigates 

IRS.gov 



RS 




Types in "IRS free file" 



RS 





P19 selected "About the Free File 
Program" while attempting to 
locate the list of software offers. 



Task 1 Path 1b 


Types in "IRS free tax filing" 



P14 navigated away from the 
Free File Landing Page and 
eventually entered their zip and 
state to review Authorized IRS 
e-File Providers..."Oh wait. 
There's people's names...This is 
confusing. I don't know what 
this means...[What were you 
looking for?]...I'm looking for 
authorized providers....[but] I 
don't want a person because I'd 
have to pay something." 


Figure 21. Observed Taxpayer Paths to Reach Free File. 

Improve Free File landing page and Software Offers page. 

Both the heuristic review of the Free File-related IRS.gov pages and the taxpayer usability 
testing revealed areas for improvement. On the Free File landing page, some taxpayers had 
trouble finding the button to start Free File or were attracted to the Fillable Forms section; the 
heuristic review pointed out that this “call to action” is difficult to find and requires the user to 
scroll down. The heuristic review found that the software offers page in particular was too heavy 
on text, some of which was unfamiliar to taxpayers in context, especially “software lookup tool.” 
The usability study validated this, as taxpayers commented on the heavy amount of text and 
feeling that there were too many options on the software offers page. They also did not see the 
software lookup tool or often did not understand what it was for. 

Improve error prevention on IRS.gov and in Free File member systems. 

The heuristic review determined room for improvement in helping prevent user errors and 
helping users correct their errors. Many Free File member systems received scores on the “error 
prevention” heuristic that reflect need for improving this capability; for example, some member 
sites had warning and error messages that were difficult to tell apart and could leave users 
wondering what action to take. The heuristic review ofIRS.gov also noted that there is no 
method of checks and balances on taxpayers’ self-determined AGI and EITC eligibility; the 
taxpayer usability testing validated the importance of this finding, given that many taxpayers in 
the usability study expressed the belief that the lookup tool would not let them report erroneous 
EITC eligibility (“At some point it will check my work”). Taxpayers in the usability study also 
struggled with the error prevention and messaging used for creating account names and 
passwords, and some systems prompted users with alerts (e.g., to check their filing status) that 
taxpayers interpreted as evidence that they had made an error. 
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Recommendations 

6.0 Recommendations 


Based on this comprehensive review of the program, the interviews with the IRS and FFA, 
and the findings from each of the workstreams, MITRE developed recommendations to 
improve the integrity and effectiveness of the Free File program. The recommended actions 
are categorized into recommended programmatic improvements to Free File, MOU-related 
recommendations, and usability recommendations based on direct taxpayer experience. Some 
key points for the IRS to consider when looking at the recommendations are: 

• While IRS has an abundance of data that can identify the demographics for those 
in the eligible population who don’t use the program, what they don’t have is the 
research to tell them why those taxpayers don’t use the program. It may be lack of 
awareness, but it is likely that isn’t the only factor. Studies to determine the “why” 
should be conducted before any other major decisions are made about the program. 

• Recommendations align across the different workstreams. While teams worked 
independently on their approaches and subsequent recommendations for their 
particular disciplines, when integrating the findings, many of the recommendations 
were aligned and validated from the separate workstreams. 

• MITRE recognizes that any recommendations concerning changes to the MOU 
must be negotiated with the FFA. While some of the recommendations may be 
rejected outright by the industry, it doesn’t negate our position that they would 
strengthen the program for the taxpayers using it. 


6.1 Improvements to the Free File Program 

Program recommendations address four primary issues that emerged from MITRE’s integrated 
assessment: 

1. Objectives of the Free File program have not been updated since the program’s inception. 

2. If Free File objectives are revised to focus on taxpayer awareness, more needs to be 
known about the target population and taxpayer behavior. 

3. IRS Free File program is a long-standing public-private partnership, but there is a lack of 
public awareness about what that means to the program. 

4. Rules that were put into the MOU to promote fairness among FFA members and promote 
choices for taxpayers translate into unclear presentation of offers and confusion for 
taxpayers. 

A description of each issue and MITRE’s recommended actions is provided in the sections 
below. 
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6.1.1 Define Updated Program Objectives and Create Corresponding Metrics 

Objectives of the Free File program have not been updated since the program’s inception. 

The IRS and FFA have not revisited the objectives of the program since its inception. As such, 
the objectives (reflected in the MOU) are outdated and don’t reflect leadership’s current view of 
program success. 

The program objective of providing a venue for free tax filing for 70 percent of the population 
has been met. The e-filing objective has been met. No other metrics currently exist by which the 
program office can measure its performance. 

Recommendation: 

MITRE recommends that IRS leadership, in collaboration with the FFA, define new program 
objectives and create metrics to detennine the Program Office’s performance against those 
objectives. Recognizing that the IRS and FFA will need to negotiate any formal changes to joint 
program objectives, MITRE recommends that IRS develop internal objectives and metrics that 
reflect its own priorities outside the MOU. While agreement between the IRS and FFA would be 
the ideal, translating priorities into objectives and metrics should not be contingent on that 
agreement. 

For example, according to IRS leadership, its focus for Free File is less about a target number of 
participants, and more about maximizing awareness to ensure people recognize it as an option. 

Potential metrics could include: 

• Number of companies participating, potentially targeting a specific market share 

• Actual number of Free File taxpayer candidate pool/participants (e.g., xx% of the DIY 
community minus RALs/RACs) 

• Metrics associated with compliance (e.g., offering parameters.) 

• Awareness numbers (see additional recommendations) 

• Web analysis metrics (e.g., number of clicks on the Free File link on the IRS.gov home 
page, percentage of select keyword searches that bring taxpayers to the IRS.gov landing 
page, etc.) 

6.1.2 Take Steps to Better Understand the Free File Target Taxpayer Population and 
Behavior 

If Free File objectives are revised to focus on taxpayer awareness, more needs to be known 
about the target population and taxpayer behavior. 

The IRS has collected demographic data on users of Free File and migration data of those users, 
but they do not have demographics data of the potential pool of Free File users. As shown in 
Section 3.3, this data would be helpful in targeting outreach. 
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The total population pool of 103 million taxpayers who are eligible may not represent an 
accurate picture of the pool of taxpayers who are candidates for Free File. Taxpayers make 
choices that take them out of the candidate pool. About 9 percent of taxpayers who have not 
taken themselves out of the candidate pool by (for example) using a paid preparer or choosing a 
RAC or RAL, use Free File (not taking into consideration those who choose free commercial 
offerings). There is no consensus on what would be an acceptable number. 

In addition, while each individual member receives customer feedback in various ways for their 
own Free File product, the IRS has not conducted a customer survey since 2009. 

The IRS has no access to data that would indicate whether a DIY filer paid for a return or 
received a free commercial offering. As the issue is whether a low-income taxpayer has to pay 
for filing, knowing how many taxpayers receive free filings either through Free File or a 
commercial product would give the IRS a better understanding of their target markets. 

Leadership acknowledged that the number of returns submitted through Free File have not 
increased in the same way that e-file numbers have and articulated a goal of increased awareness. 
The last awareness study was conducted in 2011. 

Paid advertising yields results. In MITRE’s web analysis of those searching for free tax filing, an 
estimated 85 percent of the traffic followed a paid search result to member sites, while the other 
15 percent of visits to member sites came from an organic search result. Searchers were more 
than five times likely to click on a paid search result. 

Recommendations: 

To better understand the Free File target taxpayer population and behavior, MITRE recommends 
the IRS: 

• Conduct data analysis of the demographics of the population who are prime candidates 
for Free File but not using it. 

• Conduct a taxpayer behavior study to better understand the factors involved in a 
taxpayer’s choice of filing methods. Understanding taxpayer motivation and choices 
could help detennine what would be an acceptable Free File number, if a number is 
needed at all. This aligns with the IRS-related priorities of the internal MITRE research 
program and will be proposed for a future research project. 

• Conduct a customer survey of Free File users specifically to the IRS experience of Free 
File (not the software they filed their return through) to use as a baseline and create an 
online survey for users to capture continuous feedback. 

• Propose, in the spirit of transparency, that the industry use an indicator on free 
commercial filings or request that FFA provide them the aggregate number of taxpayers 
who file their taxes using free commercial products each year. When developing metrics, 
this could help with understanding a more accurate picture of the population of the Free 
File candidate pool. 

• Commission an awareness study to baseline taxpayer awareness of Free File, specifically 
of targeted demographics detennined in the prior recommendation. Based on the results 
of this study—and other recommended research—conduct a targeted awareness 
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campaign to the desired Free File demographics, and include other IRS-sponsored venues 
such as low-income tax clinics (LITCs) and VITA sites, including IRS’ own website. 

• Detennine whether there is a cost benefit to using paid advertising to reach taxpayers 
searching for free tax filing. We recognize the IRS may have to seek statutory authority 
and budget-specific funding for such advertising if it was deemed beneficial. 

This is consistent with findings from taxpayer experience workstream that found taxpayers lack 
understanding of what the Free File program is and its relationship to the IRS. See 
recommendation in Section 6.3 to conduct research on Free File awareness, attending to 
different demographic groups ’ awareness. 


6.1.3 Increase Awareness of the Free File as a Long-Standing PPP 

IRS Free File Program is a long-standing public-private partnership, but there is a lack of 
public awareness about what that means to the program. 

Free File program is a partnership between the IRS and industry that requires cooperation to 
achieve success. This partnership requires a balance that serves the interests of the government 
and taxpayers, but also creates a value to the for-profit industry that provides the service. 

The IRS Free File program could benefit from providing clarity and scope around the PPP, both 
internally and externally. The purpose and scope of PPPs are not common, and awareness is key 
to success. 

Recommendation: 

MITRE recommends that a health assessment of the PPP become part of the overall metrics for 
the Free File program, using a generally accepted framework provided by external sources, such 
as that used by MITRE in this assessment. 

MITRE further recommends that the IRS Free File Program Office incorporate talking points 
addressing the unique nature of the PPP in terms of joint governance and the value a PPP 
provides to all partners as central themes when addressing critics or publicly discussing the 
program to help address this lack of awareness. 

This is consistent with findings from taxpayer experience workstream that found taxpayers 
seemed unclear on the relationship between the IRS and Free File offers. See recommendation in 
Section 6.3 to clarify the IRS’s relationship to the Free File program and the software offers. 


6.1.4 Simplify Member Offerings to Reduce Taxpayer Confusion 

Rules that were put into the MOU to promote fairness among FFA members and promote 
choices for taxpayers translate into unclear presentation of offers and confusion for 
taxpayers. 

The FFA is obligated to provide Free File to eligible taxpayers through individual commercial 
sites such that, when taken in aggregate, these services are offered to the lowest 70 percent of the 
taxpayer population, calculated using AGI. Moreover, the MOU requires that each company 
must offer services to at least 10 percent of total eligible taxpayers but to no more than 50 
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percent of taxpayers (10/50 rule)—a requirement set to level the playing field for smaller 
providers. 

While the 10/50 rule has been (and may continue to be) beneficial to FFA members to ensure 
equity, it is possible that it has outlived its usefulness. MITRE did not have adequate time, nor 
was it within scope, to do an analysis of how the 10/50 rule impacts Free Filed returns. However, 
from a taxpayer perspective, the different company offerings create confusion as a taxpayer 
navigates the various choices. 

Recommendations: 

MITRE recommends the IRS conduct an analysis/study of the 10/50 rule. This could include a 
confidential market study of the members to help understand the economic incentive to assist in 
preserving competition amongst the members and promoting taxpayer economic welfare; a small 
pilot to detennine the business impact of removing the rule; a consideration of whether differing 
rules for the two large companies would benefit or create inequity; and/or an in-depth data 
analysis of how the different offerings have impacted members’ free file numbers. 

Even if the rule is not changed, MITRE recommends simplifying the display of members’ 
offering to reduce taxpayer confusion. 

This is consistent with findings from taxpayer experience workstream that found taxpayers were 
overwhelmed by the Free File software options and by the information provided along with the 
options on IRS.gov Free File-related pages. See recommendation in Section 6.3 to take steps to 
increase ease of understanding eligibility requirements. 


6.2 Recommendations Related to the MOU 

MOU-specific recommendations include suggested changes directly to the content of the MOU 
document. Recommendations address the need for a comprehensive review and refresh of the 
current agreement; inclusion of search exclusion techniques in future MOU negotiations; and the 
results of a legal review of the document. 

Discussion of each issue and MITRE’s recommended actions are included in the sections below. 


6.2.1 Update MOU to Reflect Current Environment 

The MOU is an evolving document that is added to with each new negotiation, resulting in 
a piecemeal agreement that is out of date in some areas. 

The MOU generally has provisions added with each revision, but few are taken out. Much of the 
same language is used, not only in the objectives, but also in the technology portion, that has 
become outdated (e.g., text CAPTCHA). 

All but two of the members’ Free File offerings are the same as their commercial offerings. The 
security/privacy rules of Free File are the same as e-File, and all Free File members are also 
members of the Security Summit where the IRS works collaboratively with the industry to 
improve security. 
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MITRE’s review of the individual members’ processes for applying the Free File indicator 
(without looking at the actual coding) indicated most members used the same process that should 
place the indicator only on Free Filed returns. However, that is not always the case. 

In MITRE’s data analysis of returns marked with the Free File indicator, several thousand 
returns from one vendor indicated the presence of a RAC. The vendor noted that the IRS 
informed them of this irregularity, and they discovered a coding error had mistakenly placed the 
indicator on returns that had not been Free Filed, which was subsequently corrected. Another 
vendor also had several hundred returns with the Free File indicator on returns with AGI of more 
than $66,000. 

Recommendations: 

MITRE recommends that the IRS and the Alliance jointly: 

• Detennine new, measurable objectives for the Free File program, and draft a new MOU, 
pulling and using provisions from the current MOU that are relevant, discarding 
provisions that are not, and creating new provisions based on the current environment. 

• Simplify the MOU in its security and privacy provisions by simply requiring members to 
adhere to the e-File/Security Summit requirements and recommendations. 

• Add language to the MOU to specify conditions under which the indicator is applied to 
ensure quality control. 

Though not necessarily dependent on the MOU, MITRE also recommends the IRS develop a 
quality control method that would allow the IRS to validate returns with the Free File indicator 
were actually filed through the Free File program. 


6.2.2 Determine the IRS Position on Search Exclusion and Provide Guidance 

Search exclusion techniques are the latest new issue for consideration when entering into 
the next round of MOU negotiations. 

Five of the companies used a NOINDEX code to exclude their company’s Free File landing page 
from organic searches. They reported that their belief was that this kept them compliant with the 
MOU. However, while the MOU states the IRS will provide li nk s to their site, it does not 
prohibit taxpayers from accessing their sites directly. IRS press releases state that taxpayers can 
ONLY access Free File sites through IRS.gov. 

Recommendations: 

MITRE recommends that if the IRS wants taxpayers to go to Free File sites only through 
IRS.gov, the MOU should reflect that specifically as a provision, as opposed to the current 
reference in the preamble. 

MITRE further recommends the IRS detennine their position on the use of NOINDEX and add a 
provision to the MOU addressing that use. 

This is related to findings from taxpayer experience workstream that found taxpayers struggled 
to find the Free File landing page, including through IRS.gov. See recommendation in Section 
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6.3 to improve taxpayers ’ ability to find the Free File landing page both from IRS.gov web pages 
and directly from internet search results. 

6.2.3 Legal Analysis of MOU 

The following recommendations were made by MITRE’s general counsel based on a legal 
review of the MOU in collaboration with outside counsel, and after consultation with IRS 
counsel. A redline version of the current MOU is provided below. 


MOU Provision 

Recommendation 

Article 1 

Alphabetize all definitions; add new definitions for certain terms. 

Article 2 

Add language to restate MOU compliance obligations of FFI members and revise 
language to state that the federal government "declines" entering a marketplace, rather 
than making a binding pledge in perpetuity. 

Section 4.1.1 (ii) 

Change qualifying criterion for listing from definitive number of online returns to a 
minimum number of returns. 

Section 4.1.3 (ii) 

Make clear that non-discrimination applies only within the coverage (because the 
program, by definition, discriminates among taxpayers based upon income level). Also, 
include a covenant by members not to reject arbitrarily any taxpayer in the coverage. 

Section 4.2.4 

Provide that it is the responsibility of each member to assure that certain service 
providers adhere to established industry practices. 

Section 4.2.5 

Include a provision obligating members to not utilize or engage in any deceptive or 
unfairtrade practices. 

Section 4.3.1 

Insert a FFI self-regulation provision as a means of promoting truthfulness and accuracy 
in advertising the services. 

Section 4.3.2 

Provide an affirmative obligation for FFI and/or members to report to the IRS when 
truthfulness and accuracy in advertising the services is not being met 

Section 4.3.3 

Include a provision allowing the IRS to make referrals to the Federal Trade Commission if 
a good faith belief is held that a member might be engaging or has engaged in unfair and 
deceptive business practices. 

Section 4.4 

Include a requirement in functionality that a member's software programs provide both 
user access and guidance to taxpayers in addition to other functions. 

Section 4.5.3 

Require that members disclose prominently on their Free File landing page any 
limitations in the forms and schedules that may be needed by a taxpayer to support the 
member's free offerings. 

Section 4.6.2 

Clarify the date by which certain information regarding third-party security and privacy 
certifications must be provided to the IRS and the executive director. 

Section 4.6.6 

Require that members must not only implement, but also maintain in effect at all times, 
a CAPTCHA program. 

Section 4.6.7 

Require that access to a member's program must be available to visually impaired 
taxpayers without the use of special equipment. 

Section 4.7.1 

Require shutdown of a member's Free File website in the event of an unauthorized 
disclosure of information only if determined necessary in the sole discretion of the 
executive director or IRS in order to prevent any further unauthorized disclosure. 

Section 4.11 

Clarify that a member will be both liable for and timely pay any IRS penalties and/or 
interest resulting from an error in the member's software program calculations. 

Section 4.12 

Shift responsibility to members to assure that where servers or transmitters are located 
outside the U.S., taxpayers using the services of the member agree and sign a form 
consenting to the disclosure. 
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Section 4.14 

Require that a member remind a taxpayer upon his or her return to the member's Free 

File website of the terms for eligibility for current use of the member's services. 

Section 4.19.2 

Clarify sequencing ("waterfall") of actions to be offered to a taxpayer following an 
ineligibility notification. 

Section 4.24 

Reserve to the IRS the right to delist a successor to a member if the successor is not in 
compliance with the MOU, or to require the successor to reapply. 

Section 4.29.2 

Add a post-filing season review for compliance requiring members to provide detailed 
data regarding the use of the IRS Free File program by taxpayers on their website, 
including paid services provided to such taxpayers. 

Section 6.1.6 

Allow the IRS to delist a member upon the occurrence of any event that the IRS or the 
executive director determines in its discretion either jeopardizes the integrity of the IRS 
Free File program or the IRS Free File website. 

Section 6.1.7 

Allow the IRS to delist a member upon the FTC acceptance of an IRS referral along with 
the opening of an investigation by the FTC of a member's alleged unfair trade or 
deceptive practices. 

Article 8 

Include a covenant that members comply with applicable antitrust laws with failure to 
comply constituting grounds for delisting. 


Throughout the MOU there are references to “clear li nk s.” There should be a standard accepted 
format for the links. For example, the reference in the MOU to links as “clear” might mean bold, 
a different color when hovered over, a specific font style and height, etc. Currently, the phrase 
“clear links” in the MOU may lead to different interpretations and varied judgements as to what 
is clear to a specific individual. Recommend that clear li nk s be defined. 

If this does not currently exist, IRS leadership may want to consider a formal MOU relationship 
with the FTC to provide mutual assistance in evaluating whether certain FFI member business 
practices might be unfair and/or deceptive as it relates to the Free File program. 



REDLINE EIGHTH 


MEMORANDUM OF L 


6.3 Taxpayer Experience-Specific Recommendations 

The following recommendations are the result of the taxpayer experience workstreams. These 
recommendations are strictly from the perspective of the taxpayer as users of the Free File 
program. MITRE recognizes that some of these recommendations may be difficult or even 
impractical to implement; however, it is important for the IRS to understand how they could best 
serve the taxpayers, even if budget or other considerations might prevent them from doing so. 

Because these originated from the Taxpayer Usability Report they both complement and may be 
repetitive of the program-level recommendations. The fact that some of the same 
recommendations came forth from different workstreams underscores their validity. 

MITRE provides the following usability-centric, issue-level recommendations: 
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Standardize how Free File is referenced on IRS platforms and communications. 

Currently, different web pages at IRS.gov have different labels and titles for Free File, 
such as buttons that say, “File Your Taxes for Free” versus “Free File.” Standardize all 
references to one label or title and use that label/title in all communications about the 
program. Encourage Free File members to follow suit. This should also help taxpayers 
access Free File from web searches, as it may influence their search terms. 

o Clarify the IRS’s relationship to the Free File program and the software offers 
o Use visual cues like third-party logos on the IRS site to signal that taxpayers will go to 
the software provider website 

o Improve taxpayers’ ability to find the Free File landing page both from IRS.gov web 
pages (e.g., the home page) and directly from internet search results 

Take steps to increase ease of understanding eligibility requirements. The current 
volume and complexity of eligibility requirements may lead to taxpayer mistakes in 
selecting a Free File software offer. Clearer, simplified eligibility requirements will help 
taxpayers focus on the important infonnation to choose a Free File software offer, 
o Reduce the amount of information presented in individual boxes on the software offers 
page 

o Consider presenting eligibility criteria on the software offers page in a new format, 
such as in a grid view where all criteria are presented consistently or via tooltips 
o Include information on eligibility criteria in the results returned by the software lookup 
tool 

Ensure that taxpayers understand that determining eligibility for a software offer is 
ultimately their responsibility. Taxpayers expressed the belief that their eligibility would 
be “checked” at some point along the way, for example, by the software lookup tool. 
MITRE recommends that the IRS incorporate language on the Free File pages and lookup 
tool to help taxpayers understand that misunderstanding or misreporting eligibility criteria 
can lead to their selecting a Free File software offer for which they are not eligible. 

Eliminate or change the EITC eligibility question in the lookup tool and on the 
software offers page. MITRE recommends that: 

o If the IRS stated on the Free File site that all taxpayers eligible for EITC are eligible 
for Free File, the question could be removed from the tool. MITRE recognizes this 
would be a topic for negotiation between the IRS and FFA because of how that could 
impact some companies’ appearance in the tool, 
o If the IRS chooses to leave EITC in the lookup, MITRE recommends linking the 
question to the EITC eligibility tool, but ONLY after conducting usability testing on 
the EITC eligibility tool and addressing the results. 

Recommend suggested solutions be shared with FFA to increase usability of 
members’ tax preparation software systems. MITRE recommends that the IRS share 
the FFA member website suggested solutions within this report and findings identified in 
the taxpayer journey study with the FFA. This can help member programs increase the 
likelihood of taxpayers having a positive experience with Free File by improving look and 
feel, communication of eligibility on member home pages, and how to best walk taxpayers 

PRE-DECISIONAL - FOR INTERNAL IRS W&l USE ONLY - NOT FOR PUBLIC RELEASE 

87 



PRE-DECISIONAL - FOR INTERNAL IRS W&l USE ONLY - NOT FOR PUBLIC RELEASE 


through their online tax return as they fill it out. Recognize that the experience of using a 
particular member offer will affect how taxpayers perceive the Free File experience as a 
whole. 

Conduct research on the usability of taxpayer access and use of the Free File system. 

MITRE recommends that the IRS and Free File members conduct iterative testing to 
evaluate system design and usability. MITRE recommends that this usability testing 
include representation of different age groups, income groups, and education groups, to 
ensure that the Free File system is maximally usable for all eligible taxpayers. 

Conduct research on Free File awareness, attending to different demographic 
groups’ awareness. The present study was not designed to assess awareness of Free File 
among the taxpayer population. MITRE recommends that the IRS conduct future research 
to better understand whether taxpayers are aware of Free File, their understandings of 
what the program is, and their reasons for using or not using Free File. MITRE 
recommends including both previous Free Filers and eligible taxpayers who have not 
previously used Free File in this research. 
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Appendix A The Economics of IRS Free File 

[Provided in supplemental file] 
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Appendix B Crosswalk of External Program Reviews 

The embedded document provides a crosswalk of 2018 Free File program recommendations 
from the IRS Advisory Committee (IRSAC) and Taxpayer Advocate Service (TAS) with MITRE 
findings and recommendations. 

[Provided in supplemental file] 
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Appendix C ProPublica Articles on IRS Free File Program 

Huseman, J (2017, March 20). Filing Taxes Could Be Free and Simple. But Ft&R Block and 
Intuit Are Still Lobbying Against It. Retrieved from https ://www.propublica.org/article/filing- 
taxes-could-be-free-simple-hr-block-intuit-lobbying-against-it 

Root, T (2018, June 18). Why Are Millions Paying Online Tax Preparation Fees When They 
Don’t Need To? Retrieved from https://www.propublica.org/article/free-file-online-tax- 
preparation-fees-intuit-turbotax-h-r-block 

Elliott, J (2019, April, 9). Congress Is About to Ban the Government From Offering Free Online 
Tax Filing. Thank TurboTax. Retrieved from https://www.propublica.org/article/congress-is- 
about-to-ban-the-government-from-offering-free-online-tax-filing-thank-turbotax 

Elliott, J (2019, April 10). Bill to Limit IRS ’ Ability to Offer Free Tax Filing Service Is Getting 
New Scrutiny. Retrieved from https://www.propublica.org/article/bill-to-limit-irs-abilitv-to-offer- 
free-tax-filing-service-is-getting-new-scrutiny 

Elliott J. and Waldron L. (2019, April 22). Here’s How TurboTax Just Tricked You Into Paying 
to File Your Taxes. Retrieved from https://www.propublica.org/article/turbotax-just-tricked-vou- 
into-paying-to-file-vour-taxes 

Elliott, J. (2019, April 23). Updated: If You Paid TurboTax but Make Under $34,000, You Could 
Get a Refund. Here’s How. Retrieved from https://www.propublica.org/article/how-to-get- 
turbotax-refund 

Elliott, J. (2019, April 26). TurboTax Deliberately Hid Its Free File Page From Search Engines. 
Retrieved from https://www.propublica.org/article/turbotax-deliberatelv-hides-its-free-file-page- 
from-search-engines 

Elliott, J. (2019, May 3). Elizabeth Warren and Other Senators Call for Refunds and 
Investigations of TurboTax and H&R Block. Retrieved from 

https://www.propublica.org/article/elizabeth-warren-and-other-senators-call-for-refunds-and- 

investigations-of-turbotax-and-h-r-block 

Elliott, J. (2019, May 6). Senior IRS Leaders Launch Review of Agency’s Partnership With 
TurboTax and H&R Block. Retrieved from https://www.propublica.org/article/senior-irs-leaders- 
launch-review-of-partnership-with-turbotax-and-h-r-block 

Elliott, J. and Marco M. (2019, May 9). Listen to TurboTax Lie to Get Out of Refunding 
Overcharged Customers. Retrieved from https://www.propublica.org/article/listen-to-turbotax- 
lie-to-get-out-of-refunding-overcharged-customers 

Elliott, J. (2019, May 10). New York Regulator Launches Investigation Into TurboTax Maker 
Intuit and H&R Block. Retrieved from https://www.propublica.org/article/turbotax-maker-intuit- 
h-r-block-new-york-regulator-launches-investigation 

Elliott, J. and Tsutsumi, K. (2019, May 23). TurboTax Uses A “Military Discount” to Trick 
Troops Into Paying to File Their Taxes. Retrieved from 

https://www.propublica.org/article/turbotax-militarv-discount-trick-troops-paying-to-lile-taxes 
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Elliott, J. (2019, June 5). Congress Scraps Provision to Restrict IRS From Competing With 
TurboTax, Retrieved from https://www.propublica.org/article/congress-scraps-provision-to- 
restrict-irs-from-competing-with-turbotax 
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Appendix D Initial Assessment: Overall Search Engine Results 

As of July 2019, six Free File landing pages were not discoverable in Google and/or Bing. There 
are cases where a member used more than one of the techniques described below. 

The following are some of the most common techniques used to discourage discovery on organic 
internet searches, descriptions of those techniques, and results from MITRE’s source code 
analysis of members’ landing pages. Methods exist to block a URL from indexation that would 
be impossible for a third-party to observe and are therefore not included in this assessment. 
Additional infonnation about how major web search engines work is provided in the Appendix 
E. 


• Robots.txt “Disallow” Statements [R]: a text file, usually located in the root folder level, 
informing search bots which files and file folders can or cannot be crawled. For example, 
if a member site wanted to prevent their Free File landing page from appearing in web 
search engines, they could add the following instruction in their robots.txt file to prevent 
bots from crawling the page: Disallow: /ffi-landing-page 

Only one member explicitly used this technique to disallow search bots from crawling their Free 
File landing page. 

• Meta Robots NOINDEX [M]: a directive placed in the HEAD section of a web page 
providing instructions to bots at the page level not to include that page in the search index. 
For example, if a member site wanted to prevent their Free File landing page from 
appearing in web search engines, they would add the following instruction in the HEAD 
section of the landing page’s source code: <meta name=”robots” content=”noindex” /> 

Five sites explicitly NOINDEX 'ed their Free File landing pages. Some members have since 
changed this and allow search engines to index. 

• Rel “Canonical” tags [C]: a method of communicating which page among a set of 
duplicate pages is preferred for indexation. A page canonicalized to another page will be 
omitted from the search index. For example, if a member site has two very similar free tax 
filing product pages—one complies with the MOU and another commercial free tax filing 
product that does not comply, the member could add code in the compliant landing page 
instructing search bots to view that page as a duplicate of the commercial page. This 
would cause the search engines to exclude the MOU compliant landing page from the 
searchable index. 

Only one member used this technique and MITRE could not determine their intent. If the intent 
were to purposely divert users, the target URL would have led users to a similar but commercial 
product page. However, the member’s target URL was the home page. This may have 
contributed to the desired consequence of keeping the FFI page out of the search index, but the 
determination of inten t remains inconclusive. 

Two members did not use any of the aforementioned search avoidance techniques, however, 
their Free File landing pages were not discoverable on Google or Bing. The methods used to 
block URLs from the major web search engines—especially robots.txt and meta robots 
NOINDEX—as shown in the report are the most commonly used techniques to prevent content 
from showing up in the search indexes. However, there are other methods to prevent indexation 
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that are not specifically designed to block content but indirectly achieve that purpose. For 
example: 

• Using the rel canonical tag to inform the search engines that the page in question should 
be considered a duplicate of another page is intended to solve a different problem, but it 
too will prevent the page tagged as a duplicate from being indexed. 

• Building a page entirely with JavaScript—or previously with Flash—would also have the 
effect of making it very difficult, if not impossible, for the search hots to crawl the content 
for inclusion in the search index. 

• Both Google and Bing provide a mechanism through their respective webmaster tool 
applications to temporarily block a URL from indexation for periods of 90 days at a time. 
If website owners avail themselves of that feature, it would be impossible for someone 
outside of that organization to know this was done—unless the information were shared. 

MITRE cannot know with any degree of certainty whether these two members used this feature 
to block the site on Bing or even on Google at some point in time. The only thing that can be said 
with certainty is that the Free File landing page is not currently indexed in Bing, but the typical 
methods used to prevent indexation were not used in this case. 
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Appendix E How the Major Web Search Engines Work 

Major search engines such as Google and Bing are comprised of a variety of different types of 
search results (including image search, video search, product search, local business search, and a 
few others), but the two primary types of search results are known in the industry as organic 
search (the most important and by far the largest set of search results) and paid search. These two 
types of search results confonn to two different sets of rules that govern a web page’s inclusion 
and ranking. 

Paid Search vs Organic Search 

Paid search™ results are advertisements for which the web publisher participates in a type of 
auction that rewards high bid prices along with high relevance to the user’s search query. The 
relevance factor ensures that the advertisements are also useful to the searcher. In this set of 
results, publishers can exert greater control over their appearance in the results because the main 
governing factor is a willingness to pay for placement. Assuming the advertisements being 
purchased are relevant to the search queries associated with those advertisements, they need only 
out-bid their competitors for the most prominent positions at the top of the search results page. 

An important benefit of the paid search results section is that it appears at the very top of the list 
of search results. It is common for the first four paid results to be displayed before the list of 
organic search results begins. Any remaining paid results for a query will continue at the bottom 
of the first 10 organic search results and will then continue at the top of next page of results. 

Organic search results are considered free to website publishers in that they do not, and cannot, 
pay to appear in the search results. Their content’s inclusion and ranking are governed by 
complex algorithms that evaluate many factors 71 that fall under a few high-level categories 
including keyword relevance to the user’s search query; quality of user experience; and a concept 
known as “authority” or “trust.” Websites with high authority or trust scores will out-rank sites 
with lower authority when other factors such as relevance and user experience are comparable. 
This helps ensure that content from the most credible sources of information are given 
prominence in a search ahead of less trustworthy websites. The ability to measure trust in web 
searches was first developed by Google in 1996 with their patented PageRank algorithm 72 and 
was a significant innovation in the web search industry. 

The major search engines emphatically state that success in one of the two main forms of search 
—paid and organic—does not in any way influence success in the other. For example, a 
publisher that pays a high price for inclusion in the paid search results area cannot expect their 
content to rank highly in the organic search results section because of those payments. This 
“church and state” separation of paid and organic search is often described as being a firewall 
that separates the two to provide a high degree of confidence to website publishers that the 
organic results are driven by what is best for the searcher, not what is in the economic interest of 
the search engine. 


70 Search Enginer Watch, “A beginner’s guide to paid search,” 1 June 2018. [Online]. Available: 
https://searchenginewatch.com/2018/06/01/a-beginners-guide-to-paid-search/ 

71 Google, “How Search algorithms work.” [Online]. Available: https://www.google.com/search/howsearchworks/algorithms/ 

72 Wikipedia, “Page Rank,” 10 September 2019. [Online]. Available: https://en.wikipedia.org/wiki/PageRank 
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A More Detailed Description of Organic Search 

For any given search query, there could be hundreds, thousands or even millions of matching 
web pages in the organic section of the search results. Yet searchers will rarely go past the 
second page of search results and about 80 percent of the time, they do not go past the first 
page. 73 

Given this demand for the highest quality results on the first page, search engines must 
constantly work to provide only the highest quality, most relevant, most trusted and un¬ 
duplicated search results within the first 10 or 20 results. 

Well before a searcher performs begins, the search engine has been busy crawling the web 
looking for new information to include in its searchable index and carefully inspecting many 
details about this content that will help influence which queries this content will be considered a 
relevant match and how well it will rank against other web pages that are similarly relevant. 

• Crawling 74 is a discovery process whereby the search engine is looking for available 
content on the internet to make available to the searcher. However, discovery of content 
does not guarantee inclusion in the searchable index. 

• Indexing 75 is the process of including web pages and making them searchable by users of 
the search engine. It is a content curation process in which the search engine uses an 
algorithmic approach to determine what should be made available to searchers and what 
should be hidden or excluded. 

Consider a local public library. This library is similar to a search index in that the available 
books have gone through a curation process. While the library may contain many books, it is a 
small fraction of the total number of books ever written. 

Google has stated they are aware of trillions 76 of web pages on the internet. Yet, only a fraction 
of this number (tens or hundreds of billions of pages 77 ) is discoverable in their search engine. The 
rest of those discovered pages were intentionally excluded. Later, some of the reasons that 
govern why a piece of content is excluded from the searchable index will be discussed. 

The word “index” here is also analogous to the index you would find at the back of the book. 
Like a book’s index, the information being sought is managed through the creation of a list of all 
words found on the crawled web pages. These word lists are recorded in an enormous database 
of all the places on the internet where those words exist. 


73 Advanced Web Ranking, “Google Organic CTR History.” [Online]. Available: 
https://www.advancedwebranking.com/ctrstudy/ 

74 Google, "How Search organizes information.” [Online], Available: https://www.google.com/search/howsearchworks/crawling- 
indexing/ 

75 Google, "How Search organizes information.” [Online], Available: https://www.google.com/search/howsearchworks/crawling- 
indexing/ 

76 Proficient/digital, “How Google’s Search Results Work: Crawling, Indexing, and Ranking,” Enge, Eric, 21 December 2016. 
[Online], Available: https://blogs.perficientdigital.com/2016/12/21/how-googles-search-results-work-crawling-indexing-and- 
ranking/ 

77 WorldWideWebSize.com, “The size of the World Wide Web (The Internet),” 11 September 2019. [Online], Available: 
http s: //www. worldwidewebs ize. com/ 
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How Search Engines Discover Content 

Crawlers are software applications that are designed to perform two basic tasks: 1) ingest content 
found on a web page for storage in the search engine’s database for inspection at a later time by 
the indexer and 2) identification of hyperlinks on the web page, or in a website’s XML 
Sitemaps* (lists of web pages, videos or images that the website publisher wants the crawler to 
find, crawl, and index). 

These link lists help guide the crawler to the web pages it should look at next. The processes of 
content ingestion and link identification continue over-and-over again without end. When the 
crawler runs out of web pages to crawl, the process is started again. Pages are re-crawled to see if 
anything on the page has changed and make sure the page still exists. Given the enormity of the 
internet, at any given moment in time, a search engine may have deployed tens of thousands of 
crawlers simultaneously crawling the web. 

Robots.txt and Meta Robots 

Before a crawler begins its work at a website, one of the first things it does is look for a special 
file called the robots.txt file 78 (most commonly found at locations matching this structure: 
https://www.nameofsite.com/robots.txt ). This file is intended to provide instructions to the 
crawler regarding which web pages should not be crawled (these are pages the website publisher 
does not want included in the searchable index or pages that would consume excessive crawl 
resources for pages that should not be searchable such as a website’s site search results pages), 
how much time must pass before the crawler can request another page and where the crawler can 
find the site’s XML sitemaps 79 . 

Once a web page is identified for crawling, the crawler will look for another instruction located 
in the HEAD section of the web page (the HEAD is where meta information about the page is 
stored and is not something the typical user ever sees). The specific instruction the crawler is 
looking for is called the “meta robots” tag 80 and is a string of text that may take the following 
form: 

<META NAME="ROBOTS" CONTENT-'NOINDEX, NOFOLLOW M > 

In this example, the robots tag is instructing the crawler that it may not include this page in the 
search engine’s searchable index (NOINDEX) and that the crawler should not include the li nk s 
found on that page in its crawl queue (NOFOLLOW). If a website publisher omits this meta tag, 
the default condition is the same as explicitly allowing indexation and link following: 

<META NAME="ROBOTS" CONTENT-'INDEX, FOLLOW"> 

Other Reasons for Crawl and Index Exclusion 

There are other reasons that would prevent a crawler from crawling a URL in its crawl queue. A 
common reason is password protection. There is much content on the internet that is only 


78 Robots.txt, “The Web Robots Pages.” [Online], Available: https://www.robotstxt.org 

79 Sitemaps.org, “What are Sitemaps?” 7 February 2008. [Online]. Available: https://www.sitemaps.org 

80 Robots.txt, “About the Robots <META> tag,” 2007. [Online], Available: https://www.robotstxt.org/meta.html 
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accessible after successful authentication by a user. This can include many types of confidential 
information such an individual’s health data, an organization’s financial data, or a government’s 
most secure information. It is important that none of this information be discoverable by others 
in web search engines. 

Other reasons 81 content may be omitted from the searchable index: content depends on web 
technologies that the crawler is incapable of reading (once upon a time, content that was created 
with Flash was difficult for crawlers to parse and was often excluded from search); content is too 
slow to load (the crawler gives up waiting and moves on); or content is believed to be a duplicate 
of another web page on the site). 

Content Duplication: The Rel Canonical Tag 

When a website publishes the same content to multiple places on the site, the search engines will 
choose one to be the authoritative or preferred version (in web parlance, this page is referred to 
as the “canonical” version) and the crawler will ignore the duplicates. A website publisher can 
provide instructions to the search engine regarding which page the publisher wants to be treated 
as the canonical version through the user of the “rel canonical” tag 82 . Canonical tags also appear 
in the FIEAD section of a web page. For example, suppose the crawler arrives at the following 
URL: 

https://www.example.com/subject-name/second-instance-of-article-l 

The crawler finds the following instruction in the HEAD of that page: 

cli nk rel="canonical" href="https://www.example.com/subject-name/article-l" /> 

This instruction is telling the crawler that the page “article 1” is canonical and the web page 
“second instance of article 1” is a duplicate of “article 1.” The crawler will ignore the duplicate 
in favor of what the website has designated its preferred version. If the crawler finds no “rel 
canonical” tag, the default condition applies which is identical to explicitly setting a “self- 
referential” canonical tag. For example, the canonical tag we expect to find on the page named 
“article 1” above is: 

cli nk rel="canonical" href="https://www.example.com/subject-name/article-l" /> 

When a web page points its canonical tag to itself, it is saying “I am the canonical version of this 
content.” 

Content Redirection: When Web Pages Change their Web Address (URL) 

It is common for webpages to move to a new URL. To help users, and crawlers, find the new 
location of the webpage, the website implements one of a variety of methods that very quickly 
move the page’s visitor to the new address. These are called “redirects.” There are several types 
of redirect but they fall into two major categories: server-side and client-side. Server-side 
redirects are the more common and include the two most used redirect types: “301 Permanent” 


81 Google, “Why is my page missing from Google Search?” 2019. [Online]. Available: 
https://support.google.com/webmasters/answer/7474347 

82 Google, “Consolidate duplicate URLs,” 2019. [Online]. Available: https://support.google.com/webmasters/answer/139066 
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Redirects and “302 Temporary” Redirects. The most common client-side redirect is the “Meta 
Refresh.” 

The 301 Permanent Redirect and Meta Refresh Redirect will both work to instruct the crawler to 
replace the original URL in its searchable index with the new URL pointed to by the redirect 
instruction. The 302 Temporary Redirect com, on the other hand, causes the crawler to keep the 
old URL in its index and ignore the new URL on the other side of the redirect because the 
redirection has been designated “temporary” in nature. The implication of this is that the older 
URL is expected to return to its nonnal, “pre-redirecf ’ state. In response, the search engine keeps 
the older URL in its search index. 

Index Exclusions Apply to All Queries 

The directives described above such as robots.txt, meta robots, rel canonical, or redirects apply 
equally regardless of search query. In other words, the page is either searchable or not searchable 
for all queries. The presence of a web page in the index is not tied to any specific search queries. 

Search Ranking: How Search Engines Determine the Order of Results 

Attempting to describe all of the ranking factors that influence how search engines determine the 
order of search results for any given query would be beyond the scope of a summary search 
overview such as this one. But we can distill these factors down to the three main categories 
described earlier: keyword relevance, site credibility, and quality of user experience. 

Keyword relevance is perhaps the ranking factor most people intuitively understand about 
search. Solving this problem is at the heart of search because it, more than anything else, 
measures how well the search engine was able to help the user find what they were looking for. 
The nature of what people are searching for have been described as belonging to one of three 
types 83 : navigational, informational and transactional. 

Site credibility is driven by the quantity and quality of hyperlinks that point to a given website. A 
quality hyperlink is one that comes from another trusted website. An example of a high-quality 
li nk would occur if a website such as NIH.gov linked to another website. In this case, NIH.gov is 
considered a highly trusted source of health-related information. If they chose to link to another 
website, the credibility of the site on the receiving end of that link would receive a big bump in 
its own credibility score. 

Quality of user experience is determined by a variety of signals encountered by the crawler that 
it then translates into a user experience rating for the website. For example, if the crawler’s 
request for a page takes a long time to complete, that would be a negative quality signal. Other 
negative quality signals include things like pages cluttered with advertisements or text that is too 
small to read. The search engines are motivated to help their searchers avoid these poor-quality 
sites in favor of sites that deliver a much higher quality user experience. 


83 IBM, “A taxonomy of web search,” Broder, Anthony, IBM, Fall 2002. [Online], Available: 
http://www.cis.upenn.edu/~nenkova/Courses/cis430/p3-broder.pdf. 
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Types of Search Queries 

Search queries are the words and phrases people type into a search box in order to pull up a list 
of results. Below are commonly accepted types of search queries. 

Navigational searches are used by searchers who know where they want to go and are using the 
search engine to help get them there. For example, a user that wants to go to the IRS website 
might go to a search engine and simply search for [ irs ]. The search engine would almost 
certainly present the IRS.gov website as the very first search result (and quite possibly more 
search results from IRS.gov right below that). 

Informational searches can best be thought of someone performing some level of research about 
a subject they want to learn more about. For example, a taxpayer that wants to learn more about 
deductions they can claim on their tax return might perform a search for [ what deductions can i 
claim ]. For searches such as these, the search engine must quickly identify which web pages it 
has in its index that can best answer this question and then determine in what order to present the 
results beginning with the most trusted sources of this information. 

Transactional searches represent someone looking to perfonn a “web-mediated activity” such 
as making a purchase, downloading media, or subscribing to a news source. 

Within this universe of search activity, there is also another important distinction of search 
queries: branded vs. non-branded. This distinction can be applied to infonnational and 
transactional search queries. Navigational searches are by definition, branded searches. 

Branded searches are those search queries in which the user has specifically identified with 
which business or organization they prefer to achieve their search goal. This has the effect of 
turning an informational or transactional query into a navigational query. For example, here are 
two similar search queries, the first is non-branded, the second is branded. 

[ streaming music ] 

[ streaming music Spotify] 

In both cases, the user is searching for a streaming music service (transactional) but the first one 
leaves the door open for different competing services to appear, and the major streaming services 
do appear. The user has not yet decided which service they will use. In the case of the second 
search query, the user specifies a service which makes clear that they have decided where they 
want to go. 

As discussed earlier, the fact that organic search results are in a separate section from paid 
results, it is important to notice that Spotify competitors appear prominently above the organic 
results in the paid search section. Despite the user’s stated preference for Spotify as their music 
streaming service, the competing music services are willing to pay for the opportunity to show 
their search result to this searcher in the hopes of luring them away from a competitor and are 
successful in obtaining a presence in this valuable area of the search results page. 
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Appendix F Taxpayer Journey 

F.1 Heuristic Review of Free File Web Pages 

[Provided in supplemental fde] 


F.2 Heuristic Review of IRS.gov 

[Provided in supplemental file] 
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Appendix G Taxpayer Experience Testing Report 

[Provided in supplemental fde] 
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Appendix H Acronyms and Initials 


Acronym 

Definition 

AC 

Average Cost 

AGI 

Adjusted Gross Income 

C 

Compliant 

CAS 

Customer Account Services 

CAPTCHA 

Completely Automated Public Turing Test to Tell 

Computers and Humans Apart 

CDW 

Compliance Data Warehouse 

CPA 

Certified Public Accountant 

DIY 

Do it yourself 

ED 

Executive Director 

EITC 

Earned Income Tax Credit 

EPSS 

Electronic Products Support & Services 

ERO 

Electronic Return Originator 

ETARAS 

Electronic Tax Administration Research and Analysis 

System 

FFA 

Free File Alliance 

FFFF 

Free File Fillable Form 

FFI 

Free File, Inc. 

FFRDC 

Federally Funded Research and Development Center 

FY 

Fiscal Year 

FS 

Filing Season 

GAO 

Government Accountability Office 

IC 

Inconclusive 

ICANN 

Internet Corporation for Assigned Names and Numbers 

IRS 

Internal Revenue Service 

IRSAC 

IRS Advisory Committee 

LITC 

Low-Income Tax Clinics 

LLC 

Limited Liability Company 

MC 

Marginal Cost 

MOU 

Memorandum of Understanding 

MR 

Marginal Revenue 

NC 

Non-compliant 

OLS 

Office of Online Services 

OMB 

Office of Management and Budget 

P 

Price 

PCI 

Payment Card Industry 

PM 

Program Management 

PMO 

Program Management Organization 

PPP 

Public-Private Partnership 

PTIN 

Preparer Tax Identification Number 

Q 

Quantity 

RAC 

Refund Anticipation Check 

RAL 

Refund Anticipation Loan 

reCAPTCHA 

Google version of Completely Automated Public Turing 

Test to Tell Computers and Humans Apart 
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RRA 

Restructuring and Reform Act 

TAS 

Taxpayer Advocate Service 

TCE 

Tax Counseling for the Elderly 

TFF 

Traditional Free File 

TIGTA 

Treasury Inspector General for Tax Administration 

TP 

Taxpayer 

TY 

Tax Year 

URL 

Uniform Resource Locator 

UX 

User Experience 

VITA 

Volunteer Income Tax Assistance 

W&I 

Wage & Investment 
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